-- Logs begin at Tue 2022-07-26 16:24:42 UTC, end at Tue 2022-07-26 17:30:40 UTC. -- Jul 26 16:56:11.539129 dv-cinder-835384-17-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 16:56:11.552998 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 16:56:11 2022] *** Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: nodename: dv-cinder-835384-17-iscsi Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: machine: x86_64 Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: clock source: unix Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: pcre jit disabled Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: detected number of CPU cores: 4 Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: current working directory: / Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: detected binary path: /usr/bin/uwsgi-core Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: your processes number limit is 47680 Jul 26 16:56:11.556166 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: your memory page size is 4096 bytes Jul 26 16:56:11.557080 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: detected max file descriptor number: 2048 Jul 26 16:56:11.557080 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: lock engine: pthread robust mutexes Jul 26 16:56:11.557080 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: thunder lock: enabled Jul 26 16:56:11.557080 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 16:56:11.557080 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 16:56:11.598675 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: Python main interpreter initialized at 0x56123d5609c0 Jul 26 16:56:11.598675 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: python threads support enabled Jul 26 16:56:11.598675 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: your server socket listen backlog is limited to 100 connections Jul 26 16:56:11.598675 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: your mercy for graceful operations on workers is 90 seconds Jul 26 16:56:11.599343 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 16:56:11.599456 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: *** Operational MODE: preforking *** Jul 26 16:56:11.599703 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: *** uWSGI is running in multiple interpreter mode *** Jul 26 16:56:11.599703 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: spawned uWSGI master process (pid: 80453) Jul 26 16:56:11.599970 dv-cinder-835384-17-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 16:56:11.601355 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: spawned uWSGI worker 1 (pid: 80462, cores: 1) Jul 26 16:56:11.601355 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: spawned uWSGI worker 2 (pid: 80463, cores: 1) Jul 26 16:56:11.601355 dv-cinder-835384-17-iscsi devstack@keystone.service[80453]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 16:56:20.228299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:56:20.228299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: from cryptography.utils import int_from_bytes Jul 26 16:56:20.229611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:56:20.229611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: from cryptography.utils import int_from_bytes Jul 26 16:56:20.281719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 16:56:20.281990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 16:56:20.282128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] command line args: [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 16:56:20.282360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 16:56:20.282539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ================================================================================ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 16:56:20.282802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] admin_token = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.283052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config_dir = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.283239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.283432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] config_source = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.283614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] debug = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.283815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.284205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.284342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.284611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.284862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.285133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.285325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_config_append = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.285493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.285654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_dir = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.285832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.286069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.286318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.286568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.286886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.287185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.287185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.287792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.287792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.288134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.288134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.288371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.288552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.288735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.289032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.289384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.289961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.290143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] publish_errors = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.290367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.290601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.290779 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.291026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.291266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.291504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] standard_threads = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.291740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.291984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.292160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.292338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_journal = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.292513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_json = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.292691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_stderr = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.292867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] use_syslog = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.293128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.293396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.293641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.293908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.294151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.294465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.294728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.294982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.295222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.295528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.295718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.295969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.296234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.296514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.296751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.297099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.297355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.297597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.297860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.298123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.298376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.298614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.299147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.299404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.299657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.299898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.300132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.300367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.300609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.300862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.301146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.301359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.301627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.301879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.302170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.302480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.302694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.302932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.303171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.303409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.303643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.303876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.304109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.304341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.304580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.304833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.305110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.305353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.305708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.305848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.306024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.306279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.306513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.306754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.307016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.307291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.307572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.307775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.308016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.308272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.308509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.308749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.309019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.309273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.309514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.309755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.310534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.310778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.311061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.311346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.312068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.312068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.312068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.312443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.312571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.312826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.313979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.313979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.313979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.313979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.315368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.315715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.316068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.316381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.316719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.317074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.317391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.317704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.318014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.318322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.318625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.318934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.319232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.319527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.319825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.319976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:56:20.319976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: from cryptography.utils import int_from_bytes Jul 26 16:56:20.320810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.320934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 16:56:20.320934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: from cryptography.utils import int_from_bytes Jul 26 16:56:20.321212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.321212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.321212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.321556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.321817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.322176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.322505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.322821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.323115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.external = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.323413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.323786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.324081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.324374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.password = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.324666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] auth.token = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.324998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.325328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.325715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.326028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.326341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.326799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.327113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.327418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.327732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.328044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.328356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.328659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.329034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.329303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.329654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.329979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.330282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.330599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.330950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.331328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.331665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.331971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.332291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.332589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.332888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.333246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.333556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.333848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.334160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.334462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.334790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.335093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.335413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.335775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.336164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.336533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.336922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.337353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.337741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.338095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.340396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.340775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.341844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.342287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.342828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.343177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.343487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.343803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.344116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.344422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.344750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.345116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.345421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.345751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.346066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.346371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.346674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.346980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.347275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.347577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.347886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.348244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.352028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.353413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.354130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.354130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.354359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.354772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.355320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.355651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.356080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.356670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.357114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.357510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.357880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.358302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.358739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.359237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.359612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.359971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.360324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.360682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.361089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.361456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.361810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.362196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.362776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.363227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.363658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.364043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.364399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.364755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.365136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.365474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.365825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.366163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.366511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.366859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.367197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.367551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.367887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.368231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.368564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.368950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.369331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.369678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.369932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 16:56:20.370090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.370242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 16:56:20.370376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] command line args: [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 16:56:20.370506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.370639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 16:56:20.370772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ================================================================================ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 16:56:20.370916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.371053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] admin_token = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.371187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config_dir = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.371315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.371447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.371589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] config_source = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.371732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] debug = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.371874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.372009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.372185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.372316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.373042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.373042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.373307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.373443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.373595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.373730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.373882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_config_append = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.373882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.374121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.374252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_dir = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.374252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.374492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.374638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.374771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.374899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.375031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.375211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.375357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.375506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.375506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.375770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.375902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.376036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.376170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.376313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.376461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.376591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.376725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.376858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.377114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.377257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.377408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.377548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.377682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.377957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.378135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.378319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] publish_errors = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.378455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.378591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.378771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.378910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.379056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.379190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.379319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.379452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.379632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] standard_threads = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.379770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.379918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.380094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.380231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.380367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.380497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_journal = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.380625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.380771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_json = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.380771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_stderr = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.381057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] use_syslog = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.381188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.381323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 16:56:20.381471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.381615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.381749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.381881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.382941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.383079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.383277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.383525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.383667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.383811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.383943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.384086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.384224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.384368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.384503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.384689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.384837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.385972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.386118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.386255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.386389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.386574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.386716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.386889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.387986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.388184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.388319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.388494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.388713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.388884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.389154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.389295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.389414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.389628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.389801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.389955 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.390932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.391912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.392127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.392334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.392334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.392735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.393070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.393070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.393461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.393666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.393666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.394048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.394283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.394283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.394683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.394888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.394888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.395268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.395484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.395484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.395863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.396074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.396274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.396475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.396475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.396880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.397652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.397652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.398111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.398345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.398345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.398818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.399067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.399067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.399506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.399747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.399747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.400199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.400438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.400438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.400899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.401211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.401461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.401691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.401691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.402128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.402360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.402360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.402360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.403009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.403239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.403239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.403686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.404480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.404480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.404936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.405195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.405195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.405653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.405791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.405946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.406835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407079 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.407986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.408932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.409275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.409404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.409404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.409633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.409762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.409762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.external = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.410882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.password = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] auth.token = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.411912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.412299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.413285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.414939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.415284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.415284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.415649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.415773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.415951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.416210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.416459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.416695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.416945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.417214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.417447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.417686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.417940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.418194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.418477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.418757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.418962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.419168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.419555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.419808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.420067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.420348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.420622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.420870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.421132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.421368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.421740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.421977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.422215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.422466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.422756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.423009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.423281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.423598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.423850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.424211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.424539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.424765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.425161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.425303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.425483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.425761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.426052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.426248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.426617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.426860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.427193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.427348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.427604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.427975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.428276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.428520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.428834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.429102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.429346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.429572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.429952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.430226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.430485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.430813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.431145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.431303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.431666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.431931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.432235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.432478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.432715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.433086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.433347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.433597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.433848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.434215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.434555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.434759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.435558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.435810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.436050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.436288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.436527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.436772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.437044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.437285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.437539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.437771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.438017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.438254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.438491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.438728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.439131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.439378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.439613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.439854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.440097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.440351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.440608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.440846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.441103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.441363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.441600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.441850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.442088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.442332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.442567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.442814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.443046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.443295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.443543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.443791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.444037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.444285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.444529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.444777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.445053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.445289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.445539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.445787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.446034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.446280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.446731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.446968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.447216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.447465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.447712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.447963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.448197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.448451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.448701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.448939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.449196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.449430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.449661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.449910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.450165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.450417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.450765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.451007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.451240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.451476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.451715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.452062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.452321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.452580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.452832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.453095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.453343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.453610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.453834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.454071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.454309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.454543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.454790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.455040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.455268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.455514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.455812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.456075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.456322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.456568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.456812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.457360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.457721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.458094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.458312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.458535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.458745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.458885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.459004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.459172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.459417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.459666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.459906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.460161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.460408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.460656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.460889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.461227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.461622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 16:56:20.461859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 16:56:20.470013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.492391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.492758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.492758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.501861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.502219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.502540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.524656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.525045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.574435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.632410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.632664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.653681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.692235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.692496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.707029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.707287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.707464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.707683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.707904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.708118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.734447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.734701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.735010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.735305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.735595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.735881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.736173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.740622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.740852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.741084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.741306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.741526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.741742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.749346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.755753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.755977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.756396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.756619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.756839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.757086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.757307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.763468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.763759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.764094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.764436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.764750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.765102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.765419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.766285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.769604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.769805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.770040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.770374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.770526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.770700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.770928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 16:56:20.783833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.784097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:56:20.784379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.784584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:56:20.784859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.785146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.785407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.785663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.786387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.786667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:56:20.787080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.787350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 16:56:20.787691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.788021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.788356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.788680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.796766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.797087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.797360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.797637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.800184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.800544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.800914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.801269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.805632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.809164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.813543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.813758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.814031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.814153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.814315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.814450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.814589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.814848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.814984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.815301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.815461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.815657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.815952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.816157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.816346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.816568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.816688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.816851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.816995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.817155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.817424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.817764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.844319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.846064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.846245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.846658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.848646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.848935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.852372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.852738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.853136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.855531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.855823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.856104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.861099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.861382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.863569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.863866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.866207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.866589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.866946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.867316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.867677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.868034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.868329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.868484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.868624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.868776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.869008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.869212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.869486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.869770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.870045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.870326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.886888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.887150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.887311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.887435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.892355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.892607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.895944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.896349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.897635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.897996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.901776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.902403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.905720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.905919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.909220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.909421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.909567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.909707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.909847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.910081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.910227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.910380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.910595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.910739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.911022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.911395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.927249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.927560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.927834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.928103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.928635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.929061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.929418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.929778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.935864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.937520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.937795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.937947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.939579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.939941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.940510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.940796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.941117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.941395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.941684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.943446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.943828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.944206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.944570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.944938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.954181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.954472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.954755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.955036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.955321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.955593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.955874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.958361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.958747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.959111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.959485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.959849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.960206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.960565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.972140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.975669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.977338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.978716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.980214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.980642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.981751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.982488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.982488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.984328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.987133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.988745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.989174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.989326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.989609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.990709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:20.993724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.994016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.994304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.994585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.996832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:20.999166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:21.002400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.002612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.002883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.003161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.003953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:21.004338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.004705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.011540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:21.011840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.012128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.012280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:21.012633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.013009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.013451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.013828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.014187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.014543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.014919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.015273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.015654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.016026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.016431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.016777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.017172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.019485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 16:56:21.019778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.020057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.020334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.020624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.020904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.021205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.021478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.021870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.022151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.022426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.022712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.022979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.023239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 16:56:21.054634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WSGI app 0 (mountpoint='') ready in 10 seconds on interpreter 0x56123d5609c0 pid: 80463 (default app) Jul 26 16:56:21.058870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WSGI app 0 (mountpoint='') ready in 10 seconds on interpreter 0x56123d5609c0 pid: 80462 (default app) Jul 26 16:56:21.059727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31f762-1aa8-40a6-87dc-271d598847f6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:21.060635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31f762-1aa8-40a6-87dc-271d598847f6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:21.061159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31f762-1aa8-40a6-87dc-271d598847f6 None None] PATH_INFO: `/v3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:21.065758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Tue Jul 26 16:56:21 2022] GET /identity/v3/ => generated 255 bytes in 10 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:29.251943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:29.252512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:29.252512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:29.258182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:29.264285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:29.264856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:29.264856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:29.338250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: INFO dbcounter [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] Registered counter for database keystone Jul 26 16:56:29.362598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG oslo_db.sqlalchemy.engines [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80463) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 16:56:29.370144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writer thread running {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 16:56:29.564880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG passlib.handlers.bcrypt [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80463) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 16:56:29.565303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG passlib.handlers.bcrypt [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80463) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 16:56:29.623830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:29.625662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:29.772916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 26 16:56:29.943593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:30.017024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: INFO dbcounter [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] Registered counter for database keystone Jul 26 16:56:30.037836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG oslo_db.sqlalchemy.engines [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80462) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 16:56:30.048375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writer thread running {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 16:56:30.177948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558122b-c50a-4180-833b-31e67a6ff6f6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:30.183402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:30.184070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:30.184350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:30.184622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:30.194821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:30.247115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 16:56:30.247451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d9dc4-bb44-45b7-8b6e-a5f221c8a96d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:30.258799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Tue Jul 26 16:56:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 16:56:30.265714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:30.283714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f9073-6ebf-4f39-bafe-eb96b256c419 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:30.288581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:30.289778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:30.290065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:30.290339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] PATH_INFO: `/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:30.292197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] RBAC: Authorizing `identity:get_project(project_id=f28d3ad4783445e5a27ff264d5fb4f43)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:30.346762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING keystone.common.rbac_enforcer.enforcer [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 16:56:30.347126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc9347-f939-434c-9301-643e9c95840e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:30.352263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Tue Jul 26 16:56:30 2022] GET /identity/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:32.479420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787121a-1d75-41b9-aba9-f3504ce63981 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:32.480012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787121a-1d75-41b9-aba9-f3504ce63981 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:32.480012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787121a-1d75-41b9-aba9-f3504ce63981 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:32.484936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:32.491454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:32.492104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:32.492104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:32.514500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:32.516083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:32.525387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 26 16:56:32.549009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787121a-1d75-41b9-aba9-f3504ce63981 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:32.563049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787121a-1d75-41b9-aba9-f3504ce63981 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:32.567576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd864fd-ab6e-4e4c-937c-a72d78989079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:32.568230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd864fd-ab6e-4e4c-937c-a72d78989079 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:32.568519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd864fd-ab6e-4e4c-937c-a72d78989079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:32.568796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd864fd-ab6e-4e4c-937c-a72d78989079 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:32.579428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd864fd-ab6e-4e4c-937c-a72d78989079 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:32.581516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd864fd-ab6e-4e4c-937c-a72d78989079 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:32.593706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Tue Jul 26 16:56:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 16:56:32.600446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:32.611338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b679831-dbe4-4732-ab47-2138861b3d60 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:32.616100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36748e-6163-41c6-a9c9-3b22ce4b6a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:32.616816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36748e-6163-41c6-a9c9-3b22ce4b6a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:32.617199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36748e-6163-41c6-a9c9-3b22ce4b6a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:32.617522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36748e-6163-41c6-a9c9-3b22ce4b6a84 admin admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:32.619048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36748e-6163-41c6-a9c9-3b22ce4b6a84 admin admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:32.621051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36748e-6163-41c6-a9c9-3b22ce4b6a84 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:32.623738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Tue Jul 26 16:56:32 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:34.862931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18327fe-4a7a-45dd-9066-1e3063f3b21f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:34.863604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18327fe-4a7a-45dd-9066-1e3063f3b21f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:34.863604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18327fe-4a7a-45dd-9066-1e3063f3b21f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:34.865205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:34.873727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:34.874239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:34.874714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:34.907599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:34.909703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:34.966436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbeefad-5799-4a28-89e8-1de791c560bb None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:34.966935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbeefad-5799-4a28-89e8-1de791c560bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:34.967433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbeefad-5799-4a28-89e8-1de791c560bb None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:34.969909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:34.976545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:34.976831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:34.977155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:35.025026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG passlib.handlers.bcrypt [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80462) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 16:56:35.025476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG passlib.handlers.bcrypt [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80462) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 16:56:35.064110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:35.074998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:35.077430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:35.105570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:35.151425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:35.218227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:56:35.242253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:35.268349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c024b5-acec-4632-bc8c-f475fc45eaff None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:35.323086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:35.324067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:35.324391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:35.324825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:35.344906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:35.357206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:35.359593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:56:35.360802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:56:35.368146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:35.410134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10c0d9-e07c-44bc-bf28-3592973e5d61 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:35.414369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:35.464657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:35.466975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:35.467463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:35.468200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:35.485019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:35.506169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4d873-bfbe-4596-8b94-3027f0f3e0c5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:35.563607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:35.564710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:35.565272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:35.565793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:35.567168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:35.567557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:35.567740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:35.569155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:35.570423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:35.600566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:35.602823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:35.684620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:56:35.693140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:35.696994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:35.753066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3ecdb-9914-4bb6-ae0c-3e581617017c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:35.796570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:35.797277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:35.797560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:35.797833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:35.799100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:35.800618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:35.802476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:35.809435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:35.834435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:35.854901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164e738-69cd-42b6-b342-ccc3d1671f9a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:35.908827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:35.910268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:35.910721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:35.911173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:35.948385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:35.952427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:35.974089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccdf5d-b94c-46cc-927a-6d3b3284d18a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:56:35.976250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:35.978057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:35.980040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:56:35.980040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:56:35.980040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:56:35.980040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:35.980040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:56:35.985261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:36.029506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:36.030500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:36.030852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:36.031252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:36.033043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:36.034734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:36.037161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:36.043361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:36.064170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:36.133349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de10726-c559-4b92-9699-f5f33a3bea29 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:36.148037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:36.182025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b8c2b-f3a0-4987-ac0e-5e9821361108 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:36.188417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:36.189000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:36.189399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:36.189805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:36.190594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:36.190978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:36.192659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5240da1-d724-49dd-8615-5fd04ea81230 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:36.214138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:36.237881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:36.238985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:36.239468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:36.239966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:36.241569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.domain.id=default, user.id=16f5c0d2019f40b59c659bd14cb74047) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:36.242142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:36.245081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ce41-c3bd-42b9-8060-5b635f287b50 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:36.258820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1322 bytes} [Tue Jul 26 16:56:36 2022] GET /identity/v3/role_assignments?user.id=16f5c0d2019f40b59c659bd14cb74047&scope.domain.id=default&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 233 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:38.759101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f247b4e-05b5-463a-b27e-4465ae6674f5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:38.760098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f247b4e-05b5-463a-b27e-4465ae6674f5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:38.760587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f247b4e-05b5-463a-b27e-4465ae6674f5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:38.762241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:38.768407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:38.768912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:38.769056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:38.799588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:38.800830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:38.910034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 12/23] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:38.954364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f247b4e-05b5-463a-b27e-4465ae6674f5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:39.124874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f247b4e-05b5-463a-b27e-4465ae6674f5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:39.160173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:39.189825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:39.190517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:39.190802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:39.191080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:39.205156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:39.206711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:56:39.232066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:39.233465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:39.234333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:56:39.234333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:56:39.234333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:56:39.234333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:39.234333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 12/24] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:39.242996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:39.381338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:39.428167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f345e-3216-42a0-acf1-20182bd945da None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:39.483416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:39.484172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:39.484452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:39.484725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:39.485596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:39.485900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:39.487335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc349-7d6c-4731-b54d-46c31deea724 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:39.517992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 13/25] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:39.523480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:39.681630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5806c00-6cb5-4f03-a26e-036e201d9c67 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:39.681969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5806c00-6cb5-4f03-a26e-036e201d9c67 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:39.682149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5806c00-6cb5-4f03-a26e-036e201d9c67 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:39.683832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 14/26] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:56:39.693153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:39.693551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:39.693833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:39.704869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39a1cc-a607-44b6-910a-9bca4468d014 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:39.731318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:39.733138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:39.775780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19975470-e3ed-4b7b-8c2b-8217a7ebb351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:39.776479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19975470-e3ed-4b7b-8c2b-8217a7ebb351 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:39.776774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19975470-e3ed-4b7b-8c2b-8217a7ebb351 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:39.777155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19975470-e3ed-4b7b-8c2b-8217a7ebb351 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:39.778573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19975470-e3ed-4b7b-8c2b-8217a7ebb351 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:39.781318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19975470-e3ed-4b7b-8c2b-8217a7ebb351 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:39.940986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 15/27] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:56:39.985619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:40.104654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:40.118169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 13/28] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:56:39 2022] POST /identity/v3/projects => generated 301 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:56:40.119503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3758a-b8af-4f4a-bc1f-532fc927ed96 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.119803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3758a-b8af-4f4a-bc1f-532fc927ed96 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.120088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3758a-b8af-4f4a-bc1f-532fc927ed96 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.126035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 14/29] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:40.136684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e1e0-dc26-48eb-8a17-8c75a94d3caa None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.136877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e1e0-dc26-48eb-8a17-8c75a94d3caa None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.137217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168e1e0-dc26-48eb-8a17-8c75a94d3caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.162591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeed127-625e-4dea-949e-0ad5501d7b0b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:40.180217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6168e1e0-dc26-48eb-8a17-8c75a94d3caa None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:40.182383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6168e1e0-dc26-48eb-8a17-8c75a94d3caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:40.216003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:40.217762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.218526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.219334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.220363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:40.223384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:40.225614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:40.303508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 15/30] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:40.304937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f2093-219d-4b40-8346-4a0d8ef17e3f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.305284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f2093-219d-4b40-8346-4a0d8ef17e3f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.305576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f2093-219d-4b40-8346-4a0d8ef17e3f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.307389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 16/31] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:40.323183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.323504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.323762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.358599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:40.360043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:40.486963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 17/32] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:40.487637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:40.578214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 16/33] 10.222.0.48 () {60 vars in 1131 bytes} [Tue Jul 26 16:56:39 2022] POST /identity/v3/roles => generated 219 bytes in 593 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:56:40.579266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:40.625962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:40.678570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b68cc-b4d6-40fa-acda-65a9571d2f58 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:40.748217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a20e4-1cf8-4697-a056-eefafb9b9d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:40.749492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a20e4-1cf8-4697-a056-eefafb9b9d08 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.750021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a20e4-1cf8-4697-a056-eefafb9b9d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.750351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a20e4-1cf8-4697-a056-eefafb9b9d08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.765760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a20e4-1cf8-4697-a056-eefafb9b9d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-863a20e4-1cf8-4697-a056-eefafb9b9d08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:40.772794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:40.773868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:40.774663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 18/34] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:40.781221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5e017-c96a-4005-a6a1-1902621d92d4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.781842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5e017-c96a-4005-a6a1-1902621d92d4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.781842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5e017-c96a-4005-a6a1-1902621d92d4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.783392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 19/35] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:40.789002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5e017-c96a-4005-a6a1-1902621d92d4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:40.802123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25723c-a765-4fd5-b187-a99a5d6c767f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:40.837548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:40.856141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858b093-9729-4c61-a656-4da79d1b8ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:40.857567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b093-9729-4c61-a656-4da79d1b8ff6 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.858003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b093-9729-4c61-a656-4da79d1b8ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.858907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858b093-9729-4c61-a656-4da79d1b8ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.860537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858b093-9729-4c61-a656-4da79d1b8ff6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:40.862859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858b093-9729-4c61-a656-4da79d1b8ff6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:40.887485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 17/36] 10.222.0.48 () {60 vars in 1131 bytes} [Tue Jul 26 16:56:40 2022] POST /identity/v3/roles => generated 225 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:56:40.889665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.889967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.890212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.920275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:40.923055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:40.927164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5e017-c96a-4005-a6a1-1902621d92d4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:40.971230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:40.971896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:40.972232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:40.972548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:40.973331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:40.973707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:40.975862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:40.992170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 20/37] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:40.998704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:41.002101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:41.022149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 18/38] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:41.048026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:41.128599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ce2ae-ad13-46ca-82f7-1f47f95f0ad0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:41.183127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:41.188520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.190885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701029a-ff25-42fe-8fbf-5cf4afdf7d48 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:41.192349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.192651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.194441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:41.196194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:41.199772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 21/39] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:40 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:41.205919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:41.209198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:41.244071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbcb2c-9d94-47d0-8a43-3b1fad5b3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:41.244488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcb2c-9d94-47d0-8a43-3b1fad5b3886 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.244771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcb2c-9d94-47d0-8a43-3b1fad5b3886 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.245103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbcb2c-9d94-47d0-8a43-3b1fad5b3886 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.245840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbcb2c-9d94-47d0-8a43-3b1fad5b3886 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:41.247291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbcb2c-9d94-47d0-8a43-3b1fad5b3886 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:41.266568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 19/40] 10.222.0.48 () {60 vars in 1131 bytes} [Tue Jul 26 16:56:41 2022] POST /identity/v3/roles => generated 223 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:56:41.267829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ac8b-4c4a-4795-8d5c-3a60a0bd703a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.268130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ac8b-4c4a-4795-8d5c-3a60a0bd703a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.268407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97ac8b-4c4a-4795-8d5c-3a60a0bd703a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.269843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 20/41] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:41.278476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.279128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.279540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.308667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:41.310620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:41.311177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:41.353752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16965d-d38f-45c9-bf27-abe8fa5f9f76 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:41.415199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:41.416017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.416376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.416665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.418140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:41.419842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:41.422280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 22/42] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:41.429575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:41.432422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 21/43] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:56:41.458301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:41.577278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285fa8-4c5b-45bb-ae60-7d2c2930c820 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:41.607433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011800c-7b11-479d-9e18-c4393fd39bdd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:41.610716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:41.621780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:41.622922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.623309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.623642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] PATH_INFO: `/v3/domains/default/users/16f5c0d2019f40b59c659bd14cb74047/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.626444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=16f5c0d2019f40b59c659bd14cb74047, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:41.628195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:41.643312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:41.649589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 23/44] 10.222.0.48 () {58 vars in 1361 bytes} [Tue Jul 26 16:56:41 2022] PUT /identity/v3/domains/default/users/16f5c0d2019f40b59c659bd14cb74047/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:56:41.653624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b52e25-1ff5-4889-acc7-3e6dbd523694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:41.655008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b52e25-1ff5-4889-acc7-3e6dbd523694 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.655294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b52e25-1ff5-4889-acc7-3e6dbd523694 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.655650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b52e25-1ff5-4889-acc7-3e6dbd523694 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.657069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b52e25-1ff5-4889-acc7-3e6dbd523694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:41.658663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b52e25-1ff5-4889-acc7-3e6dbd523694 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:41.660489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 22/45] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:41.666978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:41.792009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921b3e6-d93c-4048-a47d-a06f8a458b13 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:41.836782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6294c6-8a81-47d8-a27e-1a289f4ee858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:41.837399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6294c6-8a81-47d8-a27e-1a289f4ee858 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:41.837683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6294c6-8a81-47d8-a27e-1a289f4ee858 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:41.837961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6294c6-8a81-47d8-a27e-1a289f4ee858 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:41.838698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6294c6-8a81-47d8-a27e-1a289f4ee858 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:41.840207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6294c6-8a81-47d8-a27e-1a289f4ee858 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:41.871438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 24/46] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:56:41 2022] POST /identity/v3/projects => generated 312 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:56:43.775297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaae1d-0ecf-49f0-809c-35d97c1f4ec4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:43.775904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaae1d-0ecf-49f0-809c-35d97c1f4ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:43.775904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aaae1d-0ecf-49f0-809c-35d97c1f4ec4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:43.777268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 23/47] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:43.784285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:43.784860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:43.784860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:43.804850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:43.806152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:43.871094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:43.896614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 25/48] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:56:43.921115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaae1d-0ecf-49f0-809c-35d97c1f4ec4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:44.050526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aaae1d-0ecf-49f0-809c-35d97c1f4ec4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:44.092121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:44.092870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:44.093146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:44.093395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:44.106621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:44.112431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:44.113916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:56:44.115163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 24/49] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:56:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:44.120417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:44.238843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bf845-68a5-4256-900e-beb61e2d6ac8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:44.286106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:44.286745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:44.287031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:44.287305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:44.288053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:44.288360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:44.289848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:44.301568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 26/50] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:44.314226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:44.356320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:44.478716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08629d76-6206-4622-bbbd-72b50284fc16 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:44.526794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:44.527590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:44.528122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:44.528556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:44.530852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:44.532694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:44.534640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 25/51] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:44 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:44.543070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:44.627973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:44.734674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afe1e8-4564-4e8b-91d7-10157a5a61b3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:44.800640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dacf0d-7356-40df-b625-a5776ce5718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:44.801725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dacf0d-7356-40df-b625-a5776ce5718c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:44.802194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dacf0d-7356-40df-b625-a5776ce5718c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:44.802661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dacf0d-7356-40df-b625-a5776ce5718c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:44.804696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dacf0d-7356-40df-b625-a5776ce5718c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:44.807421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dacf0d-7356-40df-b625-a5776ce5718c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:44.810529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 27/52] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:44.817256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:44.956147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:44.959689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa35df1-dfbf-43a2-b9e7-2fd0c6eb63ad None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:45.002201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:45.002965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:45.003261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:45.003549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:45.004383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.domain.id=default, user.id=16f5c0d2019f40b59c659bd14cb74047) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:45.004697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:45.006264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff731814-db98-4b66-9808-2bc9ae025171 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:45.013384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 26/53] 10.222.0.48 () {56 vars in 1322 bytes} [Tue Jul 26 16:56:44 2022] GET /identity/v3/role_assignments?user.id=16f5c0d2019f40b59c659bd14cb74047&scope.domain.id=default&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 530 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:45.070471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1066c-6e16-4abd-9bb1-0c31b4650051 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:45.070967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1066c-6e16-4abd-9bb1-0c31b4650051 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:45.071642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1066c-6e16-4abd-9bb1-0c31b4650051 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:45.074463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 28/54] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:45.083538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:45.083996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:45.084394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:45.108587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:45.110210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:45.207338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 27/55] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:45.238187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1066c-6e16-4abd-9bb1-0c31b4650051 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:45.378070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1066c-6e16-4abd-9bb1-0c31b4650051 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:45.391942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:45.422078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95541b0-d7db-4f30-a13f-0fd21a49d7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:45.422866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95541b0-d7db-4f30-a13f-0fd21a49d7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:45.423429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95541b0-d7db-4f30-a13f-0fd21a49d7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:45.423712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95541b0-d7db-4f30-a13f-0fd21a49d7e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:45.425300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95541b0-d7db-4f30-a13f-0fd21a49d7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:45.427105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95541b0-d7db-4f30-a13f-0fd21a49d7e5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:45.429057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 29/56] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:45.435304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:45.542945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:45.551397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a3050-4a5c-4e25-89f4-682b86dae524 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:45.592270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7187a5-cd0d-4b35-bfd0-471762865a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:45.593158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7187a5-cd0d-4b35-bfd0-471762865a92 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:45.593449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7187a5-cd0d-4b35-bfd0-471762865a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:45.593724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7187a5-cd0d-4b35-bfd0-471762865a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:45.594441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7187a5-cd0d-4b35-bfd0-471762865a92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:45.595961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7187a5-cd0d-4b35-bfd0-471762865a92 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:45.630215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 28/57] 10.222.0.48 () {60 vars in 1140 bytes} [Tue Jul 26 16:56:45 2022] POST /identity/v3/projects => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:56:47.793206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8aadf-b8de-48b7-b7bd-2ff8c06b5644 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:47.793963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8aadf-b8de-48b7-b7bd-2ff8c06b5644 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:47.793963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8aadf-b8de-48b7-b7bd-2ff8c06b5644 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:47.796322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 30/58] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:56:47.802333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:47.802872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:47.802872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:47.823693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:47.824920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:47.905383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 29/59] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:47.929330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8aadf-b8de-48b7-b7bd-2ff8c06b5644 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:48.047257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8aadf-b8de-48b7-b7bd-2ff8c06b5644 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:48.088059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a70d2-dd28-466c-9c53-6878af98adfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:48.088724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a70d2-dd28-466c-9c53-6878af98adfa None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:48.089039 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a70d2-dd28-466c-9c53-6878af98adfa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:48.089320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a70d2-dd28-466c-9c53-6878af98adfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:48.090561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a70d2-dd28-466c-9c53-6878af98adfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:48.092683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a70d2-dd28-466c-9c53-6878af98adfa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:48.094570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 31/60] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:48.101086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:48.193230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:48.221724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1370889-affd-4fe7-b44d-00f8d12dbe0f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:48.260672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e8dde-7cda-46f2-8017-bd27d9d75b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:48.261348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e8dde-7cda-46f2-8017-bd27d9d75b78 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:48.261631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e8dde-7cda-46f2-8017-bd27d9d75b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:48.261905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e8dde-7cda-46f2-8017-bd27d9d75b78 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:48.262616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e8dde-7cda-46f2-8017-bd27d9d75b78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:48.265145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e8dde-7cda-46f2-8017-bd27d9d75b78 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:48.295017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 30/61] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:56:48 2022] POST /identity/v3/users => generated 275 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:56:52.118763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a310cfa-1c78-402a-8261-6da4cbefb133 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.119838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a310cfa-1c78-402a-8261-6da4cbefb133 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.120342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a310cfa-1c78-402a-8261-6da4cbefb133 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.122023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 32/62] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:52.127424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7702a9-6b06-4fe5-9a01-20de69e8137b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.129545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.130060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.130539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.133873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7702a9-6b06-4fe5-9a01-20de69e8137b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.134424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7702a9-6b06-4fe5-9a01-20de69e8137b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.136112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 31/63] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:56:52.146208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.147112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.147919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.164841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:52.166114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:52.186175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:52.190843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:52.268029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:52.328863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 33/64] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:52.335067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 32/65] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:52.354476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:52.386556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:52.542585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:52.609677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdf3b4-493a-433e-9028-11da30360ce5 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:52.635320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1576-9542-4d25-ae40-48bcd162779b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:52.653225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:52.654878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.655323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.655753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.669090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:52.674958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:52.676065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:52.676877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:52.679255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 34/66] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:56:52.687787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:52.715076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91558c9-1eae-4cd0-9e88-9f462f6650d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:52.716009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91558c9-1eae-4cd0-9e88-9f462f6650d7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.718487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91558c9-1eae-4cd0-9e88-9f462f6650d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.718984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91558c9-1eae-4cd0-9e88-9f462f6650d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.740046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91558c9-1eae-4cd0-9e88-9f462f6650d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d91558c9-1eae-4cd0-9e88-9f462f6650d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:52.746700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:52.748364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 33/67] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:52.753540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af78646-e6d2-43df-8246-148022fc7bb9 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.754237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af78646-e6d2-43df-8246-148022fc7bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.754237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af78646-e6d2-43df-8246-148022fc7bb9 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.754237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 34/68] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:52.756228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:52.760503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af78646-e6d2-43df-8246-148022fc7bb9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:52.846459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ec996-273b-4ea1-9f51-9bb8028c6630 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:52.895161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:52.896192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.896685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.897194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.899014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:52.899578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:52.901263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8530e-7f58-4bda-afc9-142acf785f5d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:52.914347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 35/69] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:52.916008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773810-a5c5-49be-a36a-e4dc6ef67d2b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:52.916521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773810-a5c5-49be-a36a-e4dc6ef67d2b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:52.917054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773810-a5c5-49be-a36a-e4dc6ef67d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:52.945556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-35773810-a5c5-49be-a36a-e4dc6ef67d2b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:52.947986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-35773810-a5c5-49be-a36a-e4dc6ef67d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:52.952851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af78646-e6d2-43df-8246-148022fc7bb9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:52.985527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:53.001916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:53.002556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.002841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.003119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.003855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:53.004165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:53.005664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:53.017969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 35/70] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:56:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:53.023516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:53.068618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 36/71] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:53.083766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0eae2-1b11-4395-a299-71e41c15ec16 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.083766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0eae2-1b11-4395-a299-71e41c15ec16 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.083766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0eae2-1b11-4395-a299-71e41c15ec16 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.083766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 37/72] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:53.083766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0eae2-1b11-4395-a299-71e41c15ec16 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:53.125011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:53.202311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50590e5a-ce2a-4aba-ba32-7f833f681286 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:53.212068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0eae2-1b11-4395-a299-71e41c15ec16 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:53.250942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13aacd-eef9-45ea-bc29-7cc14f5c380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:53.251609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13aacd-eef9-45ea-bc29-7cc14f5c380f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.251895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13aacd-eef9-45ea-bc29-7cc14f5c380f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.252170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13aacd-eef9-45ea-bc29-7cc14f5c380f None admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.253445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13aacd-eef9-45ea-bc29-7cc14f5c380f None admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:53.254939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13aacd-eef9-45ea-bc29-7cc14f5c380f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:53.256604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 36/73] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:53.257816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774a9c4-afdf-45a5-aded-6b687055ac25 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.258105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774a9c4-afdf-45a5-aded-6b687055ac25 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.258410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774a9c4-afdf-45a5-aded-6b687055ac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.260288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:53.261499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.261897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.262793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.278063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e774a9c4-afdf-45a5-aded-6b687055ac25 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:53.280291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e774a9c4-afdf-45a5-aded-6b687055ac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:53.289262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:53.291091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:53.293295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 38/74] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:53.294726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:53.349394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:53.377915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 37/75] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:53.379258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667c0a-7ce6-411f-bd96-d5aa54eb4480 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.379545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667c0a-7ce6-411f-bd96-d5aa54eb4480 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.379817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee667c0a-7ce6-411f-bd96-d5aa54eb4480 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.385025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 38/76] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:53.385025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667c0a-7ce6-411f-bd96-d5aa54eb4480 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:53.424470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:53.442887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8af08-e781-4cc0-a155-55171cdda2d7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:53.489417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:53.490429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.490932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.491423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.506573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:53.512335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:53.514267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:56:53.515702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:53.520767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 39/77] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:53.521529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:53.542678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee667c0a-7ce6-411f-bd96-d5aa54eb4480 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:53.589768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac12a2b-54df-4871-9a0a-578a036ca498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:53.590538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac12a2b-54df-4871-9a0a-578a036ca498 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.590982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac12a2b-54df-4871-9a0a-578a036ca498 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.591449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac12a2b-54df-4871-9a0a-578a036ca498 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.651693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd12e8e-fddc-4438-84c9-d60d4d7e6949 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:53.675254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:53.708673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac12a2b-54df-4871-9a0a-578a036ca498 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:53.710392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac12a2b-54df-4871-9a0a-578a036ca498 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:53.710796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:53.711527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.711895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.712300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.736851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:53.812688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:53.815232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:53.864479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 39/78] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:56:53.864479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:53.864479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:53.864479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:53.887734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:53.889950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:53.960975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 40/79] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:56:53.961883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:53.977951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 40/80] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:53.978839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.002103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:54.042616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:54.124995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569cf4b-ed1e-4b3b-8cb6-842dfd71585b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.125303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a445b3-708b-4c08-9d79-8aa4d19d562a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.166260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.166906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.167189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.167468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.168202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:54.168507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.169171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.171179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.171349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.171501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.171501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.172235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.173782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.175481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 41/81] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:54.176284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.180786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 41/82] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:56:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:54.181493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.284331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:54.300563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bf036-c189-4fd1-be89-ac5104fe57ef None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.315888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a64c1-efe8-4ea0-9d00-e6079bf38dbf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.320015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:54.344248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.344900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.345208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.345481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.346709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.348190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.349976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 42/83] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:54.351061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.363125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.364275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.364798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.365356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.367599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.370496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.380581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 42/84] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:54.380581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.480811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0a081-4a2f-47eb-9b43-52977e0c34f3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.526358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4616a-8c1a-4b17-89b9-b65a39120960 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.528696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.529575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.529948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.530310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.547774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:54.553345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:54.555108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:54.556656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 43/85] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:54.558678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.577873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.578672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.579081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.579452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.580244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.581905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.603141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:54.615391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 43/86] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:56:54 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:56:54.616884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.624253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:54.715932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482d64-52d0-4193-bafa-c64ede1fc5fc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.775721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.776858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.777359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.777834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.780723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.781413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b707e-30fa-4e70-94e6-d1cf615327be None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.783630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.794487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 44/87] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:54.794487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.844470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:54.846264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:54.846264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:54.846264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:54.847118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=16f5c0d2019f40b59c659bd14cb74047) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:54.847627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:54.850072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:54.860261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 44/88] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/role_assignments?user.id=16f5c0d2019f40b59c659bd14cb74047&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:54.862803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:54.986906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba636b7f-dfe5-46f8-abfe-53fe3eb25d42 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:54.988838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:55.014750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:55.070255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4afbcc-06bb-4a4e-8340-b53f23477359 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:55.084415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:55.086290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:55.089221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:55.090294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:55.093199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=a4e2d0e5269b4930bf2676c185ddad57) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:55.094057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:55.098105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:55.112464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 45/89] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/role_assignments?user.id=a4e2d0e5269b4930bf2676c185ddad57&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:55.114393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:55.151431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:55.152607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:55.153096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:55.153554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:55.155947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:55.156495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:55.159533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:55.183975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 45/90] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:56:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:55.192680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:55.334051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f176c-f24d-450f-b23e-2e617def59ad None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:55.444811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:55.445686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:55.448342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:55.448796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:55.520392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d81b5c-7d09-44be-98dc-01f6e44e9298 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:55.583468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:55.633507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:55.670578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:55.672272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:55.675427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:55.675427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:55.676156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:55.677086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:55.679859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:55.683283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:55.687522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 46/91] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:55 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:55.694745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:55.974805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f6fa0-629c-4855-a15a-202222f73a7a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:55.986504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 46/92] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:56:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 873 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:56:55.992143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:56.090294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:56.092069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:56.094998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:56.097476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:56.197785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:56.247091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:56.271259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e112f-525c-48a6-be1e-01136dc4cb07 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:56.284429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:56.288826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:56.400994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:56.400994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:56.400994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:56.400994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:56.412976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:56.412976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:56.412976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 47/93] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:56:55 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:56.418934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:56.592348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb991c-fd19-4c76-b9d8-e14d70cfa591 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:56.629296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 47/94] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:56:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 936 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:56:56.634807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:56.635467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:56.635755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:56.636032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:56.636853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] RBAC: Adding query filter params (role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=a4e2d0e5269b4930bf2676c185ddad57) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:56.637193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:56.639377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:56.645095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:56.653005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 48/95] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:56:56 2022] GET /identity/v3/role_assignments?user.id=a4e2d0e5269b4930bf2676c185ddad57&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=55b0174dc86246af94379fcc1ef68040 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:56.874103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:56.891759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e862a8-1d4d-46b9-b623-ea50d3e05123 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:56.944187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3dd4d-5b86-4b2a-aea5-468c72233dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:56.945259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3dd4d-5b86-4b2a-aea5-468c72233dcf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:56.945774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3dd4d-5b86-4b2a-aea5-468c72233dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:56.946329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3dd4d-5b86-4b2a-aea5-468c72233dcf None admin] PATH_INFO: `/v3/projects/80936ef8279e48a4bd857fd4857bd529` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:56.948111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3dd4d-5b86-4b2a-aea5-468c72233dcf None admin] RBAC: Authorizing `identity:get_project(project_id=80936ef8279e48a4bd857fd4857bd529)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:56.949940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3dd4d-5b86-4b2a-aea5-468c72233dcf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:56.953247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 48/96] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:56:56 2022] GET /identity/v3/projects/80936ef8279e48a4bd857fd4857bd529 => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:56.959671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:56.973666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:57.117235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e89186-6e13-457c-92ac-b8bf9482f46b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:57.166734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:57.167277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:57.167566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:57.167851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:57.168670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=80936ef8279e48a4bd857fd4857bd529, user.id=a4e2d0e5269b4930bf2676c185ddad57) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:57.169019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:57.171197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8891f-345e-40da-883e-5fa589453ee5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:57.177914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 49/97] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:56:56 2022] GET /identity/v3/role_assignments?user.id=a4e2d0e5269b4930bf2676c185ddad57&scope.project.id=80936ef8279e48a4bd857fd4857bd529&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:56:58.396389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb41f0-d73f-451e-aba4-c3f9c869e408 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.396970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb41f0-d73f-451e-aba4-c3f9c869e408 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.397050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb41f0-d73f-451e-aba4-c3f9c869e408 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.398500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 49/98] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:56:58.406041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce929-c246-476d-a6ce-fac2fea14733 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.406507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce929-c246-476d-a6ce-fac2fea14733 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.406583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce929-c246-476d-a6ce-fac2fea14733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.434724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ea3ce929-c246-476d-a6ce-fac2fea14733 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:58.436554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea3ce929-c246-476d-a6ce-fac2fea14733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:58.535237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:58.540333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 50/99] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:58.577855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb41f0-d73f-451e-aba4-c3f9c869e408 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:58.777590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f7ac-91da-43e7-8a90-c2923c24844c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.778434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f7ac-91da-43e7-8a90-c2923c24844c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.779588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f7ac-91da-43e7-8a90-c2923c24844c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.785796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 51/100] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:58.795533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86591bc2-57b7-46b7-bbff-1fa34064f25c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.796172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86591bc2-57b7-46b7-bbff-1fa34064f25c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.796709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86591bc2-57b7-46b7-bbff-1fa34064f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.822544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb41f0-d73f-451e-aba4-c3f9c869e408 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:58.834364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-86591bc2-57b7-46b7-bbff-1fa34064f25c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:58.836849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-86591bc2-57b7-46b7-bbff-1fa34064f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:58.874182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc8528-5328-461e-af68-e15d09e16f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:58.874863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc8528-5328-461e-af68-e15d09e16f87 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.875149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc8528-5328-461e-af68-e15d09e16f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.875425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc8528-5328-461e-af68-e15d09e16f87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.888979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc8528-5328-461e-af68-e15d09e16f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e3fc8528-5328-461e-af68-e15d09e16f87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:58.895108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:58.897089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 50/101] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:58.898866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df2021-956d-49d3-8dbf-2f0ea30ce0cf None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.900295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df2021-956d-49d3-8dbf-2f0ea30ce0cf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.900295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df2021-956d-49d3-8dbf-2f0ea30ce0cf None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.902491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 51/102] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:58.905489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3aff6-2f62-4bdf-97ec-6988a27c511c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.905701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3aff6-2f62-4bdf-97ec-6988a27c511c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.905967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3aff6-2f62-4bdf-97ec-6988a27c511c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.907877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 52/103] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:56:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:56:58.914668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.915386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.915999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.943351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:58.946437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:58.969594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 52/104] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:56:58.971358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cbc49-d507-451f-b4af-d63357419575 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:58.972092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cbc49-d507-451f-b4af-d63357419575 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:58.972432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cbc49-d507-451f-b4af-d63357419575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:58.986607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:58.996838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-087cbc49-d507-451f-b4af-d63357419575 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:56:58.998517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-087cbc49-d507-451f-b4af-d63357419575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:56:59.091631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 53/105] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:59.093643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cbc49-d507-451f-b4af-d63357419575 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.144063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 53/106] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:56:59.147032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.235929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:59.320091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cbc49-d507-451f-b4af-d63357419575 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:59.358390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82214729-5b3f-4f4b-8c40-b12f11031d89 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:59.385945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:59.387263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:59.387919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:59.388667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:59.390326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:56:59.391153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:59.393934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:59.410132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 54/107] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:56:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:59.420725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.432351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:59.434873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:59.438862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:59.439697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:59.442427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:59.445941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:59.449634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 54/108] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:56:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:59.451881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.491869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:59.541869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e0ddf-68ff-408e-993e-cbfcd0ad7202 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:59.569370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:59.586013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:59.586706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:59.587041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:59.587312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:59.602766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:59.609037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:59.610572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 55/109] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:59.612072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.706514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345925d-e604-435c-a6a8-873d7b2b621d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:59.765276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:59.766177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:59.766466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:59.766731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:59.779724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:56:59.782378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931abdd-668a-45dd-aec2-4c639d8e5dd9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:56:59.785547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:56:59.787239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:56:59.788735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:56:59.791677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 55/110] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:56:59.792786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.856249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:59.858052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:59.858979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:59.859493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] PATH_INFO: `/v3/users/9cdff21797b64fa48dad9016ac350406` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:59.884488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] RBAC: Authorizing `identity:get_user(user_id=9cdff21797b64fa48dad9016ac350406)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:59.890185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:56:59.894014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 56/111] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:56:59 2022] GET /identity/v3/users/9cdff21797b64fa48dad9016ac350406 => generated 289 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:56:59.895646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:56:59.906353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:59.933103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cdc71-b2a4-45b7-be00-b12a5b74f709 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:56:59.958362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:56:59.980056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:56:59.981380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:56:59.982107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:56:59.982859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:56:59.983817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:56:59.985739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.022842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 56/112] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:56:59 2022] POST /identity/v3/projects => generated 302 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:57:00.025063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.036528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74922d-8f38-4a30-b5e9-49f710208e16 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:00.078232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:00.078888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.079227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.079495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.080286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:00.080592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:00.082500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.096048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 57/113] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:56:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:00.101210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.182887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381398b-48b3-4f45-8aa2-8b9f227eae10 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:00.218960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:00.232398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:00.233523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.234010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.234485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.235478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:00.236685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:00.238732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.251316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe9815-a2e3-4a20-ba5b-9975f4956bee None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:00.253426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:00.253806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 57/114] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:00.260687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.295529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b0591-21eb-4c89-8582-33b9353fe678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:00.296427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b0591-21eb-4c89-8582-33b9353fe678 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.296559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b0591-21eb-4c89-8582-33b9353fe678 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.296889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b0591-21eb-4c89-8582-33b9353fe678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.382231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b0591-21eb-4c89-8582-33b9353fe678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:00.382231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b0591-21eb-4c89-8582-33b9353fe678 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.439011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3f1a-969e-48fd-903c-2539a75779ef None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:00.498362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:00.499183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.499426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.499697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.501251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:00.502619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.504384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 58/115] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:00.505264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.557362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 58/116] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:00.563226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190e1bc-2833-4a37-8aaf-03c4940f98b4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.563226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190e1bc-2833-4a37-8aaf-03c4940f98b4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.563226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190e1bc-2833-4a37-8aaf-03c4940f98b4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.563226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 59/117] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:00.563226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190e1bc-2833-4a37-8aaf-03c4940f98b4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.566935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:00.603820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:00.679876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d921f-5c25-4192-9c34-87ee8e007948 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:00.703865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190e1bc-2833-4a37-8aaf-03c4940f98b4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:00.746566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbf06e-6353-4772-ab1d-8f4535fea1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:00.747271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:00.748255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf06e-6353-4772-ab1d-8f4535fea1ac None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.748763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf06e-6353-4772-ab1d-8f4535fea1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.749299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbf06e-6353-4772-ab1d-8f4535fea1ac None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.749714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.750189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.750704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.751678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf06e-6353-4772-ab1d-8f4535fea1ac None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:00.752553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:00.754498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbf06e-6353-4772-ab1d-8f4535fea1ac None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.754874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:00.756923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 60/118] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:00.758104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.762877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 59/119] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:00.762877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08628-08da-43db-b393-6c90758e2760 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:00.762877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08628-08da-43db-b393-6c90758e2760 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:00.762877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c08628-08da-43db-b393-6c90758e2760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:00.796054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-42c08628-08da-43db-b393-6c90758e2760 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:00.798186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-42c08628-08da-43db-b393-6c90758e2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:00.890326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:00.918668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 60/120] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:00.920288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08628-08da-43db-b393-6c90758e2760 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:00.955171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fd9d6-1206-43e1-9f86-4bf97f1c4ce5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:01.012918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:01.014005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.014433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.014836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.016316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:01.018413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:01.020878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 61/121] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:01.028108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:01.056253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:01.124497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c08628-08da-43db-b393-6c90758e2760 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:01.176663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4ce8e-97c2-4c1b-b7c8-f61ff03dc9b8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:01.184850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062f0d2-91d0-4eef-8ea2-e0dab988ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:01.187639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062f0d2-91d0-4eef-8ea2-e0dab988ae1c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.188132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062f0d2-91d0-4eef-8ea2-e0dab988ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.188636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062f0d2-91d0-4eef-8ea2-e0dab988ae1c None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.193803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062f0d2-91d0-4eef-8ea2-e0dab988ae1c None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:01.196212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062f0d2-91d0-4eef-8ea2-e0dab988ae1c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:01.209273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 61/122] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:00 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:01.209273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b235b9-40bd-47c9-b84f-ff7f9335fced None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.209273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b235b9-40bd-47c9-b84f-ff7f9335fced None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.209273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b235b9-40bd-47c9-b84f-ff7f9335fced None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.209273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 62/123] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:01.209273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b235b9-40bd-47c9-b84f-ff7f9335fced None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:01.250001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:01.250989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.251446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.251891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/16f5c0d2019f40b59c659bd14cb74047/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.255029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bf6cf3bb44be2a505004317684749, user_id=16f5c0d2019f40b59c659bd14cb74047, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:01.257507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:01.273470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:01.293081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 62/124] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:01 2022] PUT /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/16f5c0d2019f40b59c659bd14cb74047/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:01.295345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:01.420088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b235b9-40bd-47c9-b84f-ff7f9335fced None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:01.483822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:01.503225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35251d-3e2e-45b5-80f1-70183b256bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:01.504325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35251d-3e2e-45b5-80f1-70183b256bdb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.504807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35251d-3e2e-45b5-80f1-70183b256bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.505340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35251d-3e2e-45b5-80f1-70183b256bdb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.526895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35251d-3e2e-45b5-80f1-70183b256bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-fc35251d-3e2e-45b5-80f1-70183b256bdb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:01.538154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:01.539850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:01.541330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:01.542433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555e9b4-da13-4bb0-b8f5-089fc1a39b4a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:01.542559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 63/125] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:01.547967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.547967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.547967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.585969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:01.587929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:01.615807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:01.616813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:01.617366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:01.617943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:01.619398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] RBAC: Adding query filter params (role.id=70791be9532841bf9ab672f441e5b4b6, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=9cdff21797b64fa48dad9016ac350406) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:01.619974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:01.623043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:01.633676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 63/126] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:01 2022] GET /identity/v3/role_assignments?user.id=9cdff21797b64fa48dad9016ac350406&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=70791be9532841bf9ab672f441e5b4b6 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:01.635093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:01.799498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 64/127] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:01.801037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:01.819768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:01.937487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a97d-dac0-4b5f-a91b-36af22e8747b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:02.052350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:02.053440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:02.054237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:02.054727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/a4e2d0e5269b4930bf2676c185ddad57/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:02.059895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bf6cf3bb44be2a505004317684749, user_id=a4e2d0e5269b4930bf2676c185ddad57, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:02.063013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:02.100312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 64/128] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:01 2022] PUT /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/a4e2d0e5269b4930bf2676c185ddad57/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:02.107448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:02.127368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9027a-5d29-4770-9fb3-8a1f7dbed388 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:02.187648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:02.244451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:02.250672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:02.251906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:02.252596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:02.254448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:02.255349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:02.261308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:02.300000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 65/129] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:02.330152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:02.458389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e2b29-ba55-41f9-af7b-0d19ebab593a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:02.511365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:02.544308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:02.548622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:02.549020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:02.549278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:02.571193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:02.585298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:02.586886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:02.588336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:02.594158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 65/130] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:02.604513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:02.670963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a485df-6737-42c0-8bfc-6557879f3cfd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:02.771321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:02.773000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:02.773798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:02.774562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:02.783258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:02.790856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:02.796397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 66/131] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:02 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:02.802950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:02.954832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:03.062427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073a70a-4147-4c13-9f09-946dd012db0a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:03.085168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db19a16-aece-4653-a488-0c4d8d041ff4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:03.162710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06064b-f976-46c9-bfd5-f90050953e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:03.163810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06064b-f976-46c9-bfd5-f90050953e54 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:03.164302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06064b-f976-46c9-bfd5-f90050953e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:03.164791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06064b-f976-46c9-bfd5-f90050953e54 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:03.167200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06064b-f976-46c9-bfd5-f90050953e54 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:03.170952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06064b-f976-46c9-bfd5-f90050953e54 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:03.174232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 67/132] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:02 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:03.176388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c7431-094a-4ac2-a069-44a63e449f99 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:03.176879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c7431-094a-4ac2-a069-44a63e449f99 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:03.177420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c7431-094a-4ac2-a069-44a63e449f99 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:03.180036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 68/133] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:03.186913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:03.187494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c7431-094a-4ac2-a069-44a63e449f99 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:03.187661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:03.187944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:03.188295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:03.188893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:03.189224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:03.190665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c830b07-4899-42d2-a150-6cb5afed925c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:03.215436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 66/134] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 611 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:03.217232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:03.217523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:03.217784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:03.265784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:03.267038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:03.376201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:03.411364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c7431-094a-4ac2-a069-44a63e449f99 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:03.443359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:03.464748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 67/135] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:03.467241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:03.487027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:03.493048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:03.493048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:03.493048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/a4e2d0e5269b4930bf2676c185ddad57/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:03.501577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bf6cf3bb44be2a505004317684749, user_id=a4e2d0e5269b4930bf2676c185ddad57, role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:03.504865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:03.567319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 69/136] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:03 2022] PUT /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/a4e2d0e5269b4930bf2676c185ddad57/roles/55b0174dc86246af94379fcc1ef68040 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:03.568210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:03.710117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f069cb-05ea-4407-9121-71bcd56d14f7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:03.785512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:03.786875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:03.787566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:03.788477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:03.790951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:03.793933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:03.799352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 68/137] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:03 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:03.814129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:03.909607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:03.912929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4dcf8-a3a3-43f8-b965-101b1ecae1ff None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:04.044685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:04.046226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:04.047249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:04.050749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:04.054151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:04.059257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:04.063611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 70/138] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:04.074857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:04.111359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2eed5-2ccc-4bf5-8714-046087bcd401 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:04.193271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:04.194919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:04.195573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:04.196220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] PATH_INFO: `/v3/projects/80936ef8279e48a4bd857fd4857bd529` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:04.198590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] RBAC: Authorizing `identity:get_project(project_id=80936ef8279e48a4bd857fd4857bd529)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:04.203659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:04.207240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 69/139] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:03 2022] GET /identity/v3/projects/80936ef8279e48a4bd857fd4857bd529 => generated 312 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:04.218875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:04.228836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:04.380131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece0e72-6b62-4d20-8455-90522982c551 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:04.427634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355f09-5b89-4dab-b31e-ffd0f88d8d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:04.428308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355f09-5b89-4dab-b31e-ffd0f88d8d10 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:04.428585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355f09-5b89-4dab-b31e-ffd0f88d8d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:04.428938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355f09-5b89-4dab-b31e-ffd0f88d8d10 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:04.429795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f355f09-5b89-4dab-b31e-ffd0f88d8d10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:04.431330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f355f09-5b89-4dab-b31e-ffd0f88d8d10 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:04.480705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 71/140] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:57:04 2022] POST /identity/v3/users => generated 283 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:57:04.545187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc33e6f-cf85-4a13-a302-81af9955bf96 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:04.564277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:04.603988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ce16b-6f87-4fc3-9da2-31e8dab4431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:04.604703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ce16b-6f87-4fc3-9da2-31e8dab4431b None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:04.605011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ce16b-6f87-4fc3-9da2-31e8dab4431b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:04.605298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ce16b-6f87-4fc3-9da2-31e8dab4431b None admin] PATH_INFO: `/v3/projects/80936ef8279e48a4bd857fd4857bd529/users/a4e2d0e5269b4930bf2676c185ddad57/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:04.610676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ce16b-6f87-4fc3-9da2-31e8dab4431b None admin] RBAC: Authorizing `identity:create_grant(project_id=80936ef8279e48a4bd857fd4857bd529, user_id=a4e2d0e5269b4930bf2676c185ddad57, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:04.612196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ce16b-6f87-4fc3-9da2-31e8dab4431b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:04.633674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 70/141] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:04 2022] PUT /identity/v3/projects/80936ef8279e48a4bd857fd4857bd529/users/a4e2d0e5269b4930bf2676c185ddad57/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:57:04.840056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c0c70-34a2-44be-9b50-4379cd4c6a53 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:04.840759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c0c70-34a2-44be-9b50-4379cd4c6a53 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:04.841421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c0c70-34a2-44be-9b50-4379cd4c6a53 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:04.843897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 72/142] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:04.855221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:04.855887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:04.856379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:04.886353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:04.887996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:04.995973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 71/143] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:05.034896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c0c70-34a2-44be-9b50-4379cd4c6a53 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:05.171318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:05.179478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c0c70-34a2-44be-9b50-4379cd4c6a53 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:05.235580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:05.236744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:05.237342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:05.237850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:05.261521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:05.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:05.272920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:05.273732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:05.279570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 73/144] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:05.285496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:05.432997 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db588ed0-4ba9-4852-8b4b-cd0d514f5883 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:05.436596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:05.493288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:05.493978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:05.494265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:05.494546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:05.495331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:05.495640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:05.497144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22f697-d77c-4027-ba92-867adcbac105 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:05.508302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 72/145] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:05.516362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:05.769357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c009e-a7a0-4e12-9631-47d98efd72f8 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:05.769700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c009e-a7a0-4e12-9631-47d98efd72f8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:05.769891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c009e-a7a0-4e12-9631-47d98efd72f8 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:05.771427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 73/146] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:05.778028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c4c0e-eda4-4de5-920c-1f8e2c670672 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:05.783355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:05.783647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:05.783917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:05.826928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:05.829391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:05.885104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:05.886491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:05.887201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:05.887878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] PATH_INFO: `/v3/users/16f5c0d2019f40b59c659bd14cb74047` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:05.890961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] RBAC: Authorizing `identity:get_user(user_id=16f5c0d2019f40b59c659bd14cb74047)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:05.896910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:05.900214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 74/147] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:05 2022] GET /identity/v3/users/16f5c0d2019f40b59c659bd14cb74047 => generated 247 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:05.909970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:05.980293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 74/148] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:05.990811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:06.025747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:06.170883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb323b-23da-4103-a5b8-5834d4a3c5b3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:06.196402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:06.214729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f896076-5aaa-4169-ac46-0a3383417557 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:06.224943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:06.225635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.226322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.226601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.271018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e1c86-5a29-4197-9060-b3f8b06addb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:06.271812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1c86-5a29-4197-9060-b3f8b06addb8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.272104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1c86-5a29-4197-9060-b3f8b06addb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.272596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e1c86-5a29-4197-9060-b3f8b06addb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.290024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e1c86-5a29-4197-9060-b3f8b06addb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-929e1c86-5a29-4197-9060-b3f8b06addb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:06.296137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:06.297097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:06.297984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 75/149] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:06.299446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24bfa0-3c90-411f-92c0-b342f09e38f5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.299446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24bfa0-3c90-411f-92c0-b342f09e38f5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.299446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24bfa0-3c90-411f-92c0-b342f09e38f5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.300874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 76/150] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:06.306813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.307113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.307477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.317439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:06.319042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:06.332469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:06.338159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:06.353875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:06.447299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 75/151] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:57:06.448261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:06.516491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 77/152] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:06.517461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:06.586039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b2cfb-7ec4-4aa0-a923-6cf44deee32d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:06.621343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:06.630209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:06.630994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.631288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.631572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.632319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:06.632634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:06.634155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:06.645682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 76/153] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:06.646616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:06.711562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:06.770152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfafbe-36a6-4613-be3d-c9f5f8fa31cd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:06.781281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb74aa-8436-4473-b3f7-81d962c3e383 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:06.821795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:06.822932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.823415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.823888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.843099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:06.849533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:06.850457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:06.851685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:06.854302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 77/154] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:06.855558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:06.886240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:06.887386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:06.887882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:06.888530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:06.890793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:06.893723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:06.897138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 78/155] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:06 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:06.898621 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:06.941426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:06.999519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb8f4d-ce5b-4880-81fe-b737a67bf348 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:07.053758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:07.055337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:07.056198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:07.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] PATH_INFO: `/v3/users/9cdff21797b64fa48dad9016ac350406` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:07.059770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] RBAC: Authorizing `identity:get_user(user_id=9cdff21797b64fa48dad9016ac350406)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:07.071024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:07.073268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 78/156] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:06 2022] GET /identity/v3/users/9cdff21797b64fa48dad9016ac350406 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:07.074753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:07.181158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9e2d4-0749-4468-aa8b-7c530a717ba3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:07.245860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:07.246934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:07.247828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:07.248603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:07.250106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:07.251222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:07.252884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:07.268990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 79/157] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:07.270470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:07.301465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:07.464941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1ae00-dc6b-4a41-a8a6-c702c20c587d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:07.533829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:07.553363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098b16-a5b5-4b3b-9461-167b11c72a06 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:07.564764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:07.565470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:07.565758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:07.566038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:07.566901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=16f5c0d2019f40b59c659bd14cb74047) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:07.567222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:07.575158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:07.583485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 79/158] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:07 2022] GET /identity/v3/role_assignments?user.id=16f5c0d2019f40b59c659bd14cb74047&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 509 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:07.584383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:07.633941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:07.634754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:07.635053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:07.635335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:07.641994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:07.643623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:07.645575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 80/159] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:07 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:07.678055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:07.844668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95efd-1e2c-4aab-85bb-baa4fb0f33c4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:07.870340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:07.925777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfb76-3c40-4fdb-95ae-56ff91f42010 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:07.925968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213722f-88da-4d44-b42b-2bc036fad335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:07.927074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213722f-88da-4d44-b42b-2bc036fad335 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:07.927633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213722f-88da-4d44-b42b-2bc036fad335 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:07.928074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213722f-88da-4d44-b42b-2bc036fad335 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:07.934121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213722f-88da-4d44-b42b-2bc036fad335 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:07.937448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213722f-88da-4d44-b42b-2bc036fad335 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:07.940865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 80/160] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:07 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:07.942482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95131901-22d8-45b1-8098-6f18a06e5cd5 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:07.942991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95131901-22d8-45b1-8098-6f18a06e5cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:07.943405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95131901-22d8-45b1-8098-6f18a06e5cd5 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:07.945004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 81/161] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:07.949468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95131901-22d8-45b1-8098-6f18a06e5cd5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:08.031561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb0a6f-49f8-4e75-ab3d-238d40170f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:08.032725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb0a6f-49f8-4e75-ab3d-238d40170f48 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:08.033321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb0a6f-49f8-4e75-ab3d-238d40170f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:08.033821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb0a6f-49f8-4e75-ab3d-238d40170f48 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/9cdff21797b64fa48dad9016ac350406/roles/70791be9532841bf9ab672f441e5b4b6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:08.036397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb0a6f-49f8-4e75-ab3d-238d40170f48 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bf6cf3bb44be2a505004317684749, user_id=9cdff21797b64fa48dad9016ac350406, role_id=70791be9532841bf9ab672f441e5b4b6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:08.038360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb0a6f-49f8-4e75-ab3d-238d40170f48 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:08.043644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:08.076554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 81/162] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:07 2022] PUT /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749/users/9cdff21797b64fa48dad9016ac350406/roles/70791be9532841bf9ab672f441e5b4b6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:08.080332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:08.081294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:08.082948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:08.121945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:08.125011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:08.181858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95131901-22d8-45b1-8098-6f18a06e5cd5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:08.256308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:08.256995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:08.257508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:08.258151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:08.288507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 82/163] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:08.334041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:08.371177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:08.373501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:08.449489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:08.519799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:08.539179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 82/164] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:08.545093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:08.599255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe264866-d32e-48e7-8d0a-409585f6e3dd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:08.679733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:08.681817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:08.682644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:08.684081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:08.708742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:08.720205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:08.721927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:08.723430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:08.728883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 83/165] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:08.737676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:08.771008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d906-e424-4102-beff-7e35bc086317 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:08.865773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:08.866543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:08.866794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:08.867049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:08.868310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:08.869926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:08.873081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 83/166] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:08 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:08.882056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:08.927303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:08.971560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4b501-14d9-4a2a-bf9b-c6b83a003040 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:09.059977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:09.085917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:09.086591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.086885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.087175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.088124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:09.088358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:09.089892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:09.111749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 84/167] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:09.117654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:09.215390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f7b3d-a16f-4a68-95df-284f75efe37d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:09.340274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:09.340934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.341277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.341277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.343334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=a4e2d0e5269b4930bf2676c185ddad57) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:09.343468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:09.345024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf1cf-581f-470a-b34f-d722b7f4b10e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:09.363624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 84/168] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:08 2022] GET /identity/v3/role_assignments?user.id=a4e2d0e5269b4930bf2676c185ddad57&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 608 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:09.365121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9dc14-540a-4b98-9dc4-3154a97b1e7f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.365534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9dc14-540a-4b98-9dc4-3154a97b1e7f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.365826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9dc14-540a-4b98-9dc4-3154a97b1e7f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.367237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 85/169] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:09.372158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac503f-3865-4caf-bda1-620482102c89 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:09.374647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.374971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.375220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.414976 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:09.416240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:09.433887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:09.434672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.434957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.435322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.441950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:09.443495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:09.445659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 85/170] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:09 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:09.455517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:09.574213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 86/171] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:09.605428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:09.619842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:09.677690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:09.771093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fa8d9-f5f0-4b93-a16a-13c3ddacd041 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:09.831737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151461f-bd27-47d2-8ce4-73360e68864c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:09.832401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151461f-bd27-47d2-8ce4-73360e68864c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.832685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151461f-bd27-47d2-8ce4-73360e68864c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.832983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151461f-bd27-47d2-8ce4-73360e68864c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.895413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c756492-9c2c-4f48-ad6f-ac51eed14c42 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:09.928041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151461f-bd27-47d2-8ce4-73360e68864c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:09.929645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151461f-bd27-47d2-8ce4-73360e68864c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:09.986200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64fb5b-27e5-42ec-9487-cb10ea914548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:09.987365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64fb5b-27e5-42ec-9487-cb10ea914548 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:09.988025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64fb5b-27e5-42ec-9487-cb10ea914548 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:09.988641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64fb5b-27e5-42ec-9487-cb10ea914548 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:09.999746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:10.030605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64fb5b-27e5-42ec-9487-cb10ea914548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-7f64fb5b-27e5-42ec-9487-cb10ea914548 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:10.049247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:10.050299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:10.051095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:10.051709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 87/172] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:10.053947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8a4b6-5c34-4131-9f24-04b65f504732 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.054474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8a4b6-5c34-4131-9f24-04b65f504732 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.054980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8a4b6-5c34-4131-9f24-04b65f504732 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.057814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 88/173] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:10.062701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8a4b6-5c34-4131-9f24-04b65f504732 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.067074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 86/174] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:10.069954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.070241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.070517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.098626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:10.099868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:10.202658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 87/175] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:10.203615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.229292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:10.282040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8a4b6-5c34-4131-9f24-04b65f504732 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:10.324749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:10.326495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.326966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.327427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.328315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:10.328773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:10.330458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6384bc-0372-4b6f-9157-d1caf62fd6da None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:10.340953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 89/176] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:10.342998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b54a66-f370-40a3-92e4-3ed1c660530c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.343460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b54a66-f370-40a3-92e4-3ed1c660530c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.343888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b54a66-f370-40a3-92e4-3ed1c660530c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.345833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 90/177] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:10.350955 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b54a66-f370-40a3-92e4-3ed1c660530c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.404244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:10.413648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e7b60-b932-4417-9a5f-55c0760dff66 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:10.455863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:10.456487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.456744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.456935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.458218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:10.459714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:10.461526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 88/178] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:10.462357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.532638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b54a66-f370-40a3-92e4-3ed1c660530c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:10.568506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:10.574619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdd181-a4b9-4c5a-bef8-59236724c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:10.575271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdd181-a4b9-4c5a-bef8-59236724c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.575552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdd181-a4b9-4c5a-bef8-59236724c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.575946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdd181-a4b9-4c5a-bef8-59236724c8b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.588305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdd181-a4b9-4c5a-bef8-59236724c8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-cffdd181-a4b9-4c5a-bef8-59236724c8b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:10.594154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:10.595158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:10.595978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 91/179] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:10.597904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.597904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.597904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.619666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:10.620918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:10.625689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b57bad-cbdc-4b34-9e2e-baafb8b69562 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:10.677714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e38d29-4299-4f34-8616-bf4c4bd30bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:10.678384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e38d29-4299-4f34-8616-bf4c4bd30bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.678667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e38d29-4299-4f34-8616-bf4c4bd30bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.678944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e38d29-4299-4f34-8616-bf4c4bd30bb6 None admin] PATH_INFO: `/v3/users/6f6f5d71587a48f7a4a410391f75a27d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.696415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e38d29-4299-4f34-8616-bf4c4bd30bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=6f6f5d71587a48f7a4a410391f75a27d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:10.698126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e38d29-4299-4f34-8616-bf4c4bd30bb6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:10.699929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 89/180] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/users/6f6f5d71587a48f7a4a410391f75a27d => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:10.701389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aeaa9-4b26-44c9-ac2b-61b10fb85b31 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aeaa9-4b26-44c9-ac2b-61b10fb85b31 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.702068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aeaa9-4b26-44c9-ac2b-61b10fb85b31 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.703714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 90/181] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:10.708600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aeaa9-4b26-44c9-ac2b-61b10fb85b31 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.727981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 92/182] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:10.730299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:10.741421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.831008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aeaa9-4b26-44c9-ac2b-61b10fb85b31 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:10.865251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ea9a6-2a96-4fb2-8e1b-727eb6b74779 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:10.871362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:10.872002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.872288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.872614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.873477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] RBAC: Adding query filter params (role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=a4e2d0e5269b4930bf2676c185ddad57) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:10.873794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:10.875275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:10.881753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 91/183] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/role_assignments?user.id=a4e2d0e5269b4930bf2676c185ddad57&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=55b0174dc86246af94379fcc1ef68040 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:10.889330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:10.902182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:10.921644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:10.922602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.922882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.923155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.924335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:10.924647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:10.927161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb41c68-2ecd-4e07-9b1c-ec268525691b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:10.941381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 93/184] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:10.946603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:10.946914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:10.947164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:10.975213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:10.976672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:11.009108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:11.037452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f406d5a-702f-44b6-8f6e-bddc0250e480 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.078361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 94/185] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:11.083227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.083765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.084124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.084333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:11.084776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.206121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:11.207647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:11.240617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2d8d2-8ced-4634-88f7-54ccead2fe4a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.262772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:11.294019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.295132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.295554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.296016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:11.297738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:11.299803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:11.308018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 95/186] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:11.308018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.335149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:11.373201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 92/187] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:11.374681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.464268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da69440-ae9b-4719-8426-ec09e7bddfdc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.513920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.514696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.515161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.515611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] PATH_INFO: `/v3/users/a4e2d0e5269b4930bf2676c185ddad57` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:11.517295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] RBAC: Authorizing `identity:get_user(user_id=a4e2d0e5269b4930bf2676c185ddad57)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:11.519382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:11.528473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 96/188] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:11 2022] GET /identity/v3/users/a4e2d0e5269b4930bf2676c185ddad57 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:11.528473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.536314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73566f13-fd40-414e-9c85-e2d5f28faa05 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.587284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.588335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.588511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.588787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:11.602325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:11.602649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:11.608147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:11.610028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:11.611016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 93/189] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:11.615340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.675590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:11.684466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bb27d-fa28-4a53-9f1c-ba4b1cb735c2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.744378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.745064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.745345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.745627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:11.746378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:11.748501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:11.776481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d7827-99c1-4f83-9cb8-9e4094134166 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.776741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 97/190] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:57:11 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:57:11.777528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.830545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.831297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.831586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.831864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:11.833421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:11.834947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:11.836872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 94/191] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:11 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:11.844761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:11.950419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15d52-4721-453e-9fc8-06c13a323d23 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:11.969694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:11.996360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:11.998253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:11.998777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:11.999316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.034371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9985117-d16f-4c0b-be72-8464fe238334 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:12.053162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:12.100363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:12.102216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:12.105588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:12.106764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.107304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.107782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.109247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:12.109752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:12.112346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:12.129693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 95/192] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:12.132901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:12.269391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 98/193] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:12.270042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:12.287746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:12.338934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f2b0e-20e2-4708-8610-8d8d68d0bf93 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:12.403075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:12.403732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.404278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.404568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.406172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=6f6f5d71587a48f7a4a410391f75a27d) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:12.406172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:12.407434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c778731-29de-40d1-93e1-73162eaf57e8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:12.415074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 96/194] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:12 2022] GET /identity/v3/role_assignments?user.id=6f6f5d71587a48f7a4a410391f75a27d&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:12.416424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b6d3-22eb-4dec-b5b6-6b60f0ce9a59 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.416799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b6d3-22eb-4dec-b5b6-6b60f0ce9a59 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.417086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61b6d3-22eb-4dec-b5b6-6b60f0ce9a59 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.419176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 97/195] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:12.420940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b6d3-22eb-4dec-b5b6-6b60f0ce9a59 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:12.470572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ddc85-490a-43a3-bbe4-a3d8bc30ebab None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:12.483891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:12.525553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397ce77-53f2-45af-93d7-ab19c7e5b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:12.526255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397ce77-53f2-45af-93d7-ab19c7e5b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.526540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397ce77-53f2-45af-93d7-ab19c7e5b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.526822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397ce77-53f2-45af-93d7-ab19c7e5b6a4 None admin] PATH_INFO: `/v3/users/6f6f5d71587a48f7a4a410391f75a27d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.528686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397ce77-53f2-45af-93d7-ab19c7e5b6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=6f6f5d71587a48f7a4a410391f75a27d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:12.530351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397ce77-53f2-45af-93d7-ab19c7e5b6a4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:12.532790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 99/196] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:12 2022] GET /identity/v3/users/6f6f5d71587a48f7a4a410391f75a27d => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:12.534774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.535114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.535357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.586916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:12.589142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:12.620177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61b6d3-22eb-4dec-b5b6-6b60f0ce9a59 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:12.665097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:12.665771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.666051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.666327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] PATH_INFO: `/v3/projects/80936ef8279e48a4bd857fd4857bd529` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.668108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] RBAC: Authorizing `identity:get_project(project_id=80936ef8279e48a4bd857fd4857bd529)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:12.669828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:12.672029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 98/197] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:12 2022] GET /identity/v3/projects/80936ef8279e48a4bd857fd4857bd529 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:12.672502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:12.766395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 100/198] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:12.767152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:12.771551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:12.897773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be3bfa-089e-4d9b-9d72-b46fc81f60e4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:12.921864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269d83a-8101-4176-a30a-69dc142a930c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:12.925450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:12.939154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:12.939808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:12.940094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:12.940365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:12.941904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=80936ef8279e48a4bd857fd4857bd529, user.id=a4e2d0e5269b4930bf2676c185ddad57) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:12.941904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:12.943062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:12.957007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 101/199] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:12 2022] GET /identity/v3/role_assignments?user.id=a4e2d0e5269b4930bf2676c185ddad57&scope.project.id=80936ef8279e48a4bd857fd4857bd529&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:12.957007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:13.025906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:13.027470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:13.028087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:13.028390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:13.188164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:13.189767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:13.194715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:13.280520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b0e71-c032-475a-8b05-99ef3e6a833c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:13.358340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:13.360840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:13.361718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:13.362497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:13.393107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:13.412691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:13.414485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:13.416053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:13.421752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 102/200] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:13.428382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:13.551520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 99/201] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 879 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:13.557844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:13.565612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:13.740726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbb6cf-bebf-49d0-8189-5f939db153e4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:13.809314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:13.829199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:13.830687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:13.831173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:13.831669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:13.833835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:13.834752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:13.837900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:13.844308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93d431-f235-437e-aec8-eb8691679c1a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:13.859191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 103/202] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:13.870502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:13.963186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:13.964255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:13.964742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:13.967986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:13.971937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:13.974789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:13.978710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 100/203] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:13 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:14.004241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:14.128844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eada6-060c-4998-aac3-aa8913da8b14 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:14.211174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:14.212311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:14.212819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:14.213350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] PATH_INFO: `/v3/users/9cdff21797b64fa48dad9016ac350406` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:14.222539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] RBAC: Authorizing `identity:get_user(user_id=9cdff21797b64fa48dad9016ac350406)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:14.225673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:14.229479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 104/204] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:13 2022] GET /identity/v3/users/9cdff21797b64fa48dad9016ac350406 => generated 289 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:14.241162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:14.257033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:14.325808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:14.378510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0da5e-3502-45d4-9e57-820ab72724fe None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:14.388152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003119-1880-4177-bec0-82eb1e480370 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:14.478851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:14.480561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:14.481479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:14.483268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:14.483863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:14.484344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:14.485478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] RBAC: Adding query filter params (role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=6f6f5d71587a48f7a4a410391f75a27d) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:14.486074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:14.486554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:14.487185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:14.489764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:14.506006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 101/205] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:14 2022] GET /identity/v3/role_assignments?user.id=6f6f5d71587a48f7a4a410391f75a27d&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=55b0174dc86246af94379fcc1ef68040 => generated 259 bytes in 503 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:14.613298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:14.615154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:14.716134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:14.786475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 105/206] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:14.792302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:14.952725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e621c-6985-42d1-af5f-436a892735b5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:15.024442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b19b6cd-c731-4a48-9b8f-89a21fa18803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:15.025507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b6cd-c731-4a48-9b8f-89a21fa18803 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.026011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b6cd-c731-4a48-9b8f-89a21fa18803 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.026498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b19b6cd-c731-4a48-9b8f-89a21fa18803 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.031260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b6cd-c731-4a48-9b8f-89a21fa18803 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:15.034114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b19b6cd-c731-4a48-9b8f-89a21fa18803 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:15.037548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 102/207] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:14 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:15.043211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:15.202896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79571807-e1fd-4f25-b4be-6c8e5cf7548a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.203585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79571807-e1fd-4f25-b4be-6c8e5cf7548a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.204077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79571807-e1fd-4f25-b4be-6c8e5cf7548a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.207244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 103/208] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:15.212542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.213393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.213881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.227442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:15.236548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348adf2-8c6f-440c-8dd4-be3bee7e9851 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:15.254264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:15.256025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:15.311072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:15.311962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.312393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.312829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.313820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] RBAC: Adding query filter params (role.id=70791be9532841bf9ab672f441e5b4b6, scope.project.id=5e8bf6cf3bb44be2a505004317684749, user.id=9cdff21797b64fa48dad9016ac350406) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:15.315760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:15.317494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:15.338058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 106/209] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:15 2022] GET /identity/v3/role_assignments?user.id=9cdff21797b64fa48dad9016ac350406&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=70791be9532841bf9ab672f441e5b4b6 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:15.387942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 104/210] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:15.451185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:15.486021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:15.587422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ceac0-92f6-4f3b-be3b-a71009cb61be None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:15.654529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56980e4f-dcb4-4561-831c-37f290018f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:15.655660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56980e4f-dcb4-4561-831c-37f290018f58 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.656120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56980e4f-dcb4-4561-831c-37f290018f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.656605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56980e4f-dcb4-4561-831c-37f290018f58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.661956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56980e4f-dcb4-4561-831c-37f290018f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:15.663514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56980e4f-dcb4-4561-831c-37f290018f58 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:15.666536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 107/211] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:15.673299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:15.876540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d38a9-51ef-4332-b9f8-031630ae274f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:15.898859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e3e2e-3023-458b-8940-ce8de6705219 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.899706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e3e2e-3023-458b-8940-ce8de6705219 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.900170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e3e2e-3023-458b-8940-ce8de6705219 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.903572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 108/212] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:15.910782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.911264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.911756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.929104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:15.939010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9169a-d040-4161-bb59-56470b7d3b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:15.939667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9169a-d040-4161-bb59-56470b7d3b6e None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:15.939950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9169a-d040-4161-bb59-56470b7d3b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:15.940802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9169a-d040-4161-bb59-56470b7d3b6e None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:15.941588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9169a-d040-4161-bb59-56470b7d3b6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:15.943130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9169a-d040-4161-bb59-56470b7d3b6e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:15.959219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:15.962519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:15.983978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 105/213] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:57:15 2022] POST /identity/v3/users => generated 297 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:57:16.089092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3f563-eaea-4f26-b293-f174db9de15f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:16.090284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3f563-eaea-4f26-b293-f174db9de15f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:16.091326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3f563-eaea-4f26-b293-f174db9de15f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:16.095669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 106/214] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:16.105659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:16.106595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:16.107966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:16.122383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 109/215] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:16.146447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:16.150979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:16.153459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:16.173881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:16.298819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 107/216] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:16.335954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b006-fcd6-4a0e-aa49-8ef3f7ab3e7b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:16.338520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:16.393083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:16.393763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:16.394053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:16.394332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:16.415287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:16.425255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:16.426305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:16.427349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:16.430434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 110/217] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:16.437499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:16.533986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f200a-dfaf-447e-87b3-06880d6bbabf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:16.557882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:16.596405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:16.597141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519605-1fb7-4220-b079-f222df693895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:16.599022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519605-1fb7-4220-b079-f222df693895 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:16.599022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519605-1fb7-4220-b079-f222df693895 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:16.599022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72519605-1fb7-4220-b079-f222df693895 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:16.622522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c726a09-36e3-4d5b-bfa1-f054caad4533 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:16.625377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72519605-1fb7-4220-b079-f222df693895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-72519605-1fb7-4220-b079-f222df693895 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:16.634677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:16.635672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:16.636480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:16.637295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 108/218] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:16.644431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519605-1fb7-4220-b079-f222df693895 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:16.692317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:16.692890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:16.693204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:16.693478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:16.694249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:16.694555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:16.696057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:16.728866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 111/219] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:16.751143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:16.866881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72519605-1fb7-4220-b079-f222df693895 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:16.968395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49184a9-b10a-435d-b84a-acd616d3725a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:17.006516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:17.007963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:17.008482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:17.009032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:17.010015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:17.010546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:17.012306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:17.023616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 109/220] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:17.039962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:17.074821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:17.076894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:17.078635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:17.079335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] PATH_INFO: `/v3/users/6f6f5d71587a48f7a4a410391f75a27d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:17.082645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] RBAC: Authorizing `identity:get_user(user_id=6f6f5d71587a48f7a4a410391f75a27d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:17.085633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:17.088069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 112/221] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:16 2022] GET /identity/v3/users/6f6f5d71587a48f7a4a410391f75a27d => generated 283 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:17.095072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:17.137496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:17.171250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:17.329449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4162c666-07f7-4d3f-92d4-729a3e49ebca None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:17.336180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb08b3-e81d-445c-a602-bee60ae115b2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:17.394380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:17.395057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:17.395339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:17.395609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] PATH_INFO: `/v3/users/456c74c7459347448a08d2856fdf714b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:17.415756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:17.416870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:17.417409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:17.417899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:17.420737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:17.423618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:17.426964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 113/222] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:17 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:17.430263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] RBAC: Authorizing `identity:get_user(user_id=456c74c7459347448a08d2856fdf714b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:17.433199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:17.436273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:17.443302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 110/223] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:17 2022] GET /identity/v3/users/456c74c7459347448a08d2856fdf714b => generated 297 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:17.452995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:17.637957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:17.692452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b761b36-f98f-4a4b-a343-b89e29622c88 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:17.759709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:17.780020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe4efe-3988-496d-b0e1-4f15cc33cc17 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:17.806077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b3d1-4d50-4eb1-bae1-41165def4c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:17.807539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b3d1-4d50-4eb1-bae1-41165def4c05 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:17.808248 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b3d1-4d50-4eb1-bae1-41165def4c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:17.808902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b3d1-4d50-4eb1-bae1-41165def4c05 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/6f6f5d71587a48f7a4a410391f75a27d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:17.816347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b3d1-4d50-4eb1-bae1-41165def4c05 None admin] RBAC: Authorizing `identity:create_grant(project_id=20d464db83b0461b8a5f3a3fd94529c0, user_id=6f6f5d71587a48f7a4a410391f75a27d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:17.819360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b3d1-4d50-4eb1-bae1-41165def4c05 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:17.845103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:17.845765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:17.846073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:17.848640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:17.873721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 114/224] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:17 2022] PUT /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/6f6f5d71587a48f7a4a410391f75a27d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:17.971044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:17.972474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:18.076953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294cc3-ea7a-466f-8986-677289084827 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:18.077395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294cc3-ea7a-466f-8986-677289084827 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:18.077546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7294cc3-ea7a-466f-8986-677289084827 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:18.089840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 115/225] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:18 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:18.096746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:18.097158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:18.097400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:18.121442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:18.134151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 111/226] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:18.146158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:18.150377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:18.151663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:18.346551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15617046-8d86-4d13-9f92-3289a86601fe None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:18.458444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 116/227] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 16:57:18.498842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:18.499926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:18.500395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:18.500873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:18.507799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:18.515083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:18.530798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:18.534195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 112/228] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:18 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:18.548620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:18.681625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:18.762387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8351c46-ac67-4ae4-9a34-70d7a3cf3a08 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:18.781344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:18.847302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:18.848584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:18.849165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:18.849674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:18.871626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:18.883814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:18.884778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:18.885631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:18.886300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 117/229] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:18.891110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084552e1-078c-40fc-bff4-08c006ce1638 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:18.896142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:18.977633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:18.978203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:18.978495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:18.978773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:18.979638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=456c74c7459347448a08d2856fdf714b) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:18.979959 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:18.981565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:18.992355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 113/230] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:18 2022] GET /identity/v3/role_assignments?user.id=456c74c7459347448a08d2856fdf714b&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 259 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:19.078077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415e7c7-4892-43d2-ac25-153d09a17bc4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:19.118692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:19.136125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:19.137244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.137749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.138295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.139551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:19.140112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:19.141904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:19.159543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 118/231] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:19.168640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:19.312178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee503d-08a5-45f6-9a23-f1a91e81aa25 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:19.348226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:19.356365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:19.358169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.358675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.359183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] PATH_INFO: `/v3/users/6f6f5d71587a48f7a4a410391f75a27d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.360808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] RBAC: Authorizing `identity:get_user(user_id=6f6f5d71587a48f7a4a410391f75a27d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:19.362613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:19.364632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 114/232] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:19 2022] GET /identity/v3/users/6f6f5d71587a48f7a4a410391f75a27d => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:19.377925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:19.538452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045266b-cf68-453e-a60b-f570f107c272 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:19.585781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd3117-6766-4101-9f63-afd5e1c39c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:19.586464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd3117-6766-4101-9f63-afd5e1c39c39 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.586755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd3117-6766-4101-9f63-afd5e1c39c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.587026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd3117-6766-4101-9f63-afd5e1c39c39 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.588394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd3117-6766-4101-9f63-afd5e1c39c39 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:19.590107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd3117-6766-4101-9f63-afd5e1c39c39 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:19.593009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 119/233] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:19 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:19.601822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:19.609540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d18c3-81e8-49b0-8f54-4efe3f4a6ebe None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.609930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d18c3-81e8-49b0-8f54-4efe3f4a6ebe None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.610174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d18c3-81e8-49b0-8f54-4efe3f4a6ebe None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.611587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 120/234] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:19.628339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89281f-e3df-448d-8848-d440633d7034 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.628742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89281f-e3df-448d-8848-d440633d7034 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.628923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89281f-e3df-448d-8848-d440633d7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.659527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5a89281f-e3df-448d-8848-d440633d7034 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:19.660775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5a89281f-e3df-448d-8848-d440633d7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:19.753746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf24a3-2b36-4d2c-b695-b060df7efb85 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:19.787437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:19.805832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:19.806642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.806888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.807170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/6f6f5d71587a48f7a4a410391f75a27d/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.808412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 121/235] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:19.814744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] RBAC: Authorizing `identity:create_grant(project_id=20d464db83b0461b8a5f3a3fd94529c0, user_id=6f6f5d71587a48f7a4a410391f75a27d, role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:19.814744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:19.815726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941575f-e58a-4a40-8720-0593a05ed6c8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.816689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941575f-e58a-4a40-8720-0593a05ed6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.817185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941575f-e58a-4a40-8720-0593a05ed6c8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.819508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 122/236] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:19.827629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:19.828803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:19.829761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:19.832892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 115/237] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:19 2022] PUT /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/6f6f5d71587a48f7a4a410391f75a27d/roles/55b0174dc86246af94379fcc1ef68040 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:19.866425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:19.868151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:19.883603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:19.973147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 123/238] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:20.059300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:20.071853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:20.145392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b0690-1059-4d42-88d1-ea60192a480b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:20.214691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:20.215852 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:20.216850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:20.219814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:20.241655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:20.253045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:20.254918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:20.256522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 116/239] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:20.271959 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:20.373978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5b3ac-6a4d-4103-b726-c9aaa4fc5cbc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:20.430612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:20.489824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:20.490506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:20.490785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:20.491059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:20.492409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:20.494089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:20.495822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 124/240] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:20.504243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:20.504895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d6fe3-380e-49fd-ba64-87496f7762dc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:20.549675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:20.618059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:20.619060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:20.619533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:20.619993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:20.621280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:20.621799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:20.624164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:20.648737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 117/241] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:20.659281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:20.770798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcda90-907d-47db-bb13-d778f0368280 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:20.852529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:20.853208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:20.853514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:20.855838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:20.857632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:20.859330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:20.903687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 125/242] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:57:20 2022] POST /identity/v3/users => generated 293 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:57:20.915386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1672f-1a03-452d-a4d2-53c276b0c87f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:20.964072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:20.964859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:20.965178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:20.965511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] PATH_INFO: `/v3/users/7628d25ec3c84c39811980a3d4cc1c48` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:20.985497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] RBAC: Authorizing `identity:get_user(user_id=7628d25ec3c84c39811980a3d4cc1c48)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:20.987062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:20.989229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 118/243] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:20 2022] GET /identity/v3/users/7628d25ec3c84c39811980a3d4cc1c48 => generated 297 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:20.995078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:21.051005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:21.129906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91f48c-168d-478f-ac61-62b65acedd3f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:21.173734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:21.174364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:21.174653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:21.175021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:21.320928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:21.323854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:21.416465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:21.567434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 126/244] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:21.573769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:21.630905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:21.706453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615ea17-7c7c-466d-9286-21286ad1c82a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:21.751782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d804a-c19e-4cc6-ae1d-eb7ea40c5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:21.752692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d804a-c19e-4cc6-ae1d-eb7ea40c5e12 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:21.753002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d804a-c19e-4cc6-ae1d-eb7ea40c5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:21.753364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d804a-c19e-4cc6-ae1d-eb7ea40c5e12 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:21.754827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d804a-c19e-4cc6-ae1d-eb7ea40c5e12 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:21.756345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d804a-c19e-4cc6-ae1d-eb7ea40c5e12 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:21.758610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 119/245] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:21 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:21.768905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:21.958945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724d34-399b-4cdb-aaff-72569ef6c5a7 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:21.959437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724d34-399b-4cdb-aaff-72569ef6c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:21.959929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50724d34-399b-4cdb-aaff-72569ef6c5a7 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:21.963201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 120/246] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:21.976325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:21.977121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:21.977600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:21.979259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cae3e-cccd-4b89-a556-fe0bcd16bdb5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:22.019390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:22.022016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:22.031217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:22.032235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:22.032690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:22.033138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:22.034506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] RBAC: Adding query filter params (role.id=70791be9532841bf9ab672f441e5b4b6, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=7628d25ec3c84c39811980a3d4cc1c48) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:22.035029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:22.037704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb0fca-5aeb-446e-a4f0-88534cac3b85 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:22.048333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 127/247] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:21 2022] GET /identity/v3/role_assignments?user.id=7628d25ec3c84c39811980a3d4cc1c48&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=70791be9532841bf9ab672f441e5b4b6 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:22.053063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812080c-0619-43c6-a378-768143986ec2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:22.053063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812080c-0619-43c6-a378-768143986ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:22.053063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812080c-0619-43c6-a378-768143986ec2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:22.053899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 128/248] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:22.062774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:22.062994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:22.063427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:22.089254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:22.100853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:22.105431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:22.179216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 121/249] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:22.206769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:22.255911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:22.347014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf5374-e4bd-446e-be3b-44f5bc9b8701 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:22.413005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:22.413005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:22.413005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:22.413005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:22.417119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 129/250] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:22.450818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:22.460470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:22.461544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:22.462343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:22.464279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 122/251] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:22.470522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:22.513218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:22.664592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a378f5-4c9d-482d-b725-d599a783e47a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:22.804465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:22.805405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:22.805860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:22.806315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:22.807564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:22.808066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:22.809605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:22.812055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:22.819199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:22.836735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa75cf-b9eb-4979-989d-5014ab603d2a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:22.839966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 130/252] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:22.855285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:22.952919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:22.954031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:22.954603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:22.955476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:22.994745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:23.013874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:23.015763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:23.017413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 123/253] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:23.028659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:23.137111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b46a02-71af-40d1-ae5f-baf9cda1b0f8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:23.280664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:23.281922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:23.282420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:23.282910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] PATH_INFO: `/v3/users/456c74c7459347448a08d2856fdf714b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:23.285650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] RBAC: Authorizing `identity:get_user(user_id=456c74c7459347448a08d2856fdf714b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:23.289119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:23.293276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 131/254] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:22 2022] GET /identity/v3/users/456c74c7459347448a08d2856fdf714b => generated 297 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:23.301695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:23.382601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7439d-e07b-46c3-b8bb-b5546127ba28 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:23.454156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:23.454831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:23.455152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:23.455420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:23.456183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:23.456487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:23.458264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:23.476068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 124/255] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:23.485218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:23.529367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:23.567378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:23.668366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ccc3c-6736-4187-b2c7-685d55e70d74 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:23.685840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec839d08-1834-4afe-853c-434b228b9c53 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:23.740189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60728588-5b16-4c4a-b8d6-12447d815c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:23.740868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60728588-5b16-4c4a-b8d6-12447d815c74 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:23.741188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60728588-5b16-4c4a-b8d6-12447d815c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:23.741465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60728588-5b16-4c4a-b8d6-12447d815c74 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:23.743371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60728588-5b16-4c4a-b8d6-12447d815c74 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:23.744889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60728588-5b16-4c4a-b8d6-12447d815c74 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:23.746992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 132/256] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:23 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:23.748115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17add-6157-4410-ab60-fb58d13a06ef None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:23.748602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17add-6157-4410-ab60-fb58d13a06ef None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:23.749059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17add-6157-4410-ab60-fb58d13a06ef None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:23.750520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 133/257] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:23.751344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b99448-428b-44ad-8ce6-e5ddff54017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:23.752014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99448-428b-44ad-8ce6-e5ddff54017c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:23.752295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99448-428b-44ad-8ce6-e5ddff54017c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:23.752567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b99448-428b-44ad-8ce6-e5ddff54017c None admin] PATH_INFO: `/v3/users/6f6f5d71587a48f7a4a410391f75a27d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:23.758377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b99448-428b-44ad-8ce6-e5ddff54017c None admin] RBAC: Authorizing `identity:get_user(user_id=6f6f5d71587a48f7a4a410391f75a27d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:23.759951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b99448-428b-44ad-8ce6-e5ddff54017c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:23.761926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 125/258] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:23 2022] GET /identity/v3/users/6f6f5d71587a48f7a4a410391f75a27d => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:23.763819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:23.764278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:23.764662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:23.768604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17add-6157-4410-ab60-fb58d13a06ef None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:23.802491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:23.803755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:23.938374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 126/259] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:23.939222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:23.986033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:24.084105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4f663-931e-4123-a856-7bf130c1bc10 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:24.085239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17add-6157-4410-ab60-fb58d13a06ef None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:24.145538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:24.146188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:24.146467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:24.146739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:24.164060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:24.164762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:24.165072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:24.165356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/456c74c7459347448a08d2856fdf714b/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:24.171520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] RBAC: Authorizing `identity:create_grant(project_id=20d464db83b0461b8a5f3a3fd94529c0, user_id=456c74c7459347448a08d2856fdf714b, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:24.173154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:24.178889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:24.199924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 134/260] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:23 2022] PUT /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/456c74c7459347448a08d2856fdf714b/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:24.200476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:24.324738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:24.327626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:24.365005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:24.503193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb3520a-6d09-4c29-9e74-ecf1e8ae853d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:24.503448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 127/261] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:24.517018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:24.598910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:24.601160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:24.602223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:24.602979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:24.628559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:24.642269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:24.644062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:24.645701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:24.650172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 135/262] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:24.666793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:24.698834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f8283-d166-44a5-b610-42952dd99427 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:24.721544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:24.749002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:24.749002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:24.749002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:24.749002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:24.751033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:24.752571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:24.756427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 128/263] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:24 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:24.764742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:24.828752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:24.942553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4bfe5-66ba-401a-8062-ef03d544d677 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:24.969000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793021c-5a4b-4fc8-99cb-757c55ff8dff None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:25.038934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:25.039493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:25.039775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:25.040052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:25.040874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=6f6f5d71587a48f7a4a410391f75a27d) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:25.041214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:25.042727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:25.045613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:25.047198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:25.047996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:25.048759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:25.050659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 129/264] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:24 2022] GET /identity/v3/role_assignments?user.id=6f6f5d71587a48f7a4a410391f75a27d&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:25.057065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:25.057911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:25.060754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:25.082363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 136/265] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:25.088510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:25.217008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:25.259448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4419224-3c85-4fbb-92db-482628492fac None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:25.318764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925d95-8225-45da-8ed8-9171bd829d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:25.319822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925d95-8225-45da-8ed8-9171bd829d40 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:25.320314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925d95-8225-45da-8ed8-9171bd829d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:25.320801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925d95-8225-45da-8ed8-9171bd829d40 None admin] PATH_INFO: `/v3/users/6f6f5d71587a48f7a4a410391f75a27d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:25.322509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925d95-8225-45da-8ed8-9171bd829d40 None admin] RBAC: Authorizing `identity:get_user(user_id=6f6f5d71587a48f7a4a410391f75a27d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:25.324302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925d95-8225-45da-8ed8-9171bd829d40 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:25.327207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 130/266] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:25 2022] GET /identity/v3/users/6f6f5d71587a48f7a4a410391f75a27d => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:25.333672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:25.495250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4b9d1-c7ab-4cfc-8d60-f35cdfadba7d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:25.515785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:25.543971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:25.544689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:25.545058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:25.545454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:25.618515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08798246-aace-420f-9df2-449b3b112f2f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:25.619193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08798246-aace-420f-9df2-449b3b112f2f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:25.619688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08798246-aace-420f-9df2-449b3b112f2f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:25.621686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 131/267] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:25.629948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:25.630590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:25.631085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:25.633409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:25.634938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:25.673623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:25.676283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:25.861849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 137/268] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:25.868148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:25.869405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 132/269] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:25.892268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:25.950067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:26.004802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837acb0-6504-4af6-ad8b-d2fe994abec5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:26.051788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:26.053192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.053714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.054196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.055906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:26.057697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:26.062031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 138/270] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:25 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:26.070099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:26.100914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:26.193656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:26.204717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a00a9-55ca-4af8-8d47-46983a2f4db4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:26.252908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:26.254235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.254768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.255239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.256270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] RBAC: Adding query filter params (role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=6f6f5d71587a48f7a4a410391f75a27d) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:26.256793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:26.258850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5141137-2b32-4962-8f3d-eda82266662c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:26.267507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 139/271] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:26 2022] GET /identity/v3/role_assignments?user.id=6f6f5d71587a48f7a4a410391f75a27d&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=55b0174dc86246af94379fcc1ef68040 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:26.316340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ce2d4-16e8-4e98-94e5-778763e993a3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:26.355837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2bcdf-1c3a-4b12-9477-74b3b704acde None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.356482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2bcdf-1c3a-4b12-9477-74b3b704acde None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.357055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f2bcdf-1c3a-4b12-9477-74b3b704acde None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.359945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 140/272] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:26.366175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.366827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.367330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.392008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:26.393957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:26.436745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:26.437710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.438201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.438673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.481432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:26.488376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:26.489390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:26.490286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:26.498948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 133/273] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 555 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:26.507976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:26.522681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 141/274] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:26.599560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:26.715187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad6b1f-2cd0-4a43-9f1f-8ee1a1f88f7c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:26.731772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:26.745540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:26.772052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:26.773196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.773690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.774353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.775600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:26.776077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:26.778700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d855-54a2-4455-b950-a942a7f0f2c9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:26.795000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 134/275] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:26.797286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73131f86-17cf-483f-824d-8be0d09127b9 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.797694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73131f86-17cf-483f-824d-8be0d09127b9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.798114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73131f86-17cf-483f-824d-8be0d09127b9 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.800648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 135/276] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:26.804430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4624611-f923-4098-a3e0-9c629f459432 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:26.806042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73131f86-17cf-483f-824d-8be0d09127b9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:26.878834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc293ce6-93ca-4fa0-bc11-06a8c99b59c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:26.880089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc293ce6-93ca-4fa0-bc11-06a8c99b59c7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.880580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc293ce6-93ca-4fa0-bc11-06a8c99b59c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.881135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc293ce6-93ca-4fa0-bc11-06a8c99b59c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.883336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc293ce6-93ca-4fa0-bc11-06a8c99b59c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:26.886602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc293ce6-93ca-4fa0-bc11-06a8c99b59c7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:26.890065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 142/277] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:26.896735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:26.897539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:26.898658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:26.930124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:26.931342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:26.970494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73131f86-17cf-483f-824d-8be0d09127b9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:27.025114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 143/278] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:27.025114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.040228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:27.041384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.041883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.042379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] PATH_INFO: `/v3/users/fa729a22c70840b39b6473f22cbbe635` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.071217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=fa729a22c70840b39b6473f22cbbe635)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:27.074152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:27.082611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 136/279] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:26 2022] GET /identity/v3/users/fa729a22c70840b39b6473f22cbbe635 => generated 293 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:27.082611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.121405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:27.137030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:27.176399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6597da-2c45-47a6-85cc-2b597ee4daa5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:27.225984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:27.227231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.227684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.228212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.229261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:27.231647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:27.240138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d661e82-edb2-425a-b37e-33cba1ef5dc1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:27.264099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 144/280] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:57:27 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:57:27.265881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.300789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:27.301943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.302593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.303105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.318150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:27.324204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:27.325221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:27.328507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:27.333833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 137/281] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:27.342646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.469496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbafc17-be59-4ee7-98d0-ad712e02eef7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:27.501664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:27.517136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:27.521820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:27.522934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.523451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.524271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.525386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, user.id=fa729a22c70840b39b6473f22cbbe635, scope.system=all) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:27.526186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:27.527623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d78e2-8fd8-4954-8bae-af5b921d0677 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:27.528660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f2bcd-3128-4d44-bcee-e6c8c934bf50 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:27.545444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 145/282] 10.222.0.48 () {56 vars in 1308 bytes} [Tue Jul 26 16:57:27 2022] GET /identity/v3/role_assignments?user.id=fa729a22c70840b39b6473f22cbbe635&scope.system=all&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:27.594137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:27.595194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.595685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.596174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.596668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe2c5-6aa2-4c49-a859-25a1ea08a14f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.597341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe2c5-6aa2-4c49-a859-25a1ea08a14f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.597601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:27.597827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbe2c5-6aa2-4c49-a859-25a1ea08a14f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.598121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:27.600641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 146/283] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:27.600909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:27.608022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.608349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.608617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.624516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 138/284] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:27.633161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.641692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:27.643056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:27.747503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 147/285] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:27.774124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.788081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98d9a1-e81b-4d5e-be6a-3578b7eb3d2b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:27.869960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:27.870562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:27.870841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:27.871115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] PATH_INFO: `/v3/users/7628d25ec3c84c39811980a3d4cc1c48` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:27.872744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] RBAC: Authorizing `identity:get_user(user_id=7628d25ec3c84c39811980a3d4cc1c48)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:27.874673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:27.876710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 139/286] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:27 2022] GET /identity/v3/users/7628d25ec3c84c39811980a3d4cc1c48 => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:27.885104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:27.924764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:27.981179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99707e88-b363-4302-ab20-901c7bd21770 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:28.057560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:28.089811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:28.090491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:28.090879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:28.091224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:28.117062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d46f54-d309-44c3-95d3-6f02da65481d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:28.120693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:28.126007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:28.127433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:28.128534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 148/287] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:28.139990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:28.195479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:28.197697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:28.198490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:28.199322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:28.206442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:28.209714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:28.213485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 140/288] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:27 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:28.224904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:28.426011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67e9e3-8208-4619-91e0-1c6024e686e9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:28.465405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:28.489051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:28.489699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:28.491955 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950aa5b-b3bd-4c49-9204-b7964c4ee5d8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:28.493692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:28.494047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:28.494828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:28.495139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:28.498664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:28.518245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 149/289] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:28.534061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:28.590202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:28.592410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:28.593283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:28.594113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/7628d25ec3c84c39811980a3d4cc1c48/roles/70791be9532841bf9ab672f441e5b4b6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:28.599408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:28.602143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] RBAC: Authorizing `identity:create_grant(project_id=20d464db83b0461b8a5f3a3fd94529c0, user_id=7628d25ec3c84c39811980a3d4cc1c48, role_id=70791be9532841bf9ab672f441e5b4b6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:28.604210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:28.649173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 141/290] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:57:28 2022] PUT /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/users/7628d25ec3c84c39811980a3d4cc1c48/roles/70791be9532841bf9ab672f441e5b4b6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:28.681943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d9325c-13ef-42c6-bd0b-295fdb6267f8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:28.780633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:28.783246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:28.784216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:28.785110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] PATH_INFO: `/v3/users/456c74c7459347448a08d2856fdf714b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:28.790550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] RBAC: Authorizing `identity:get_user(user_id=456c74c7459347448a08d2856fdf714b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:28.794048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:28.798917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 150/291] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:28 2022] GET /identity/v3/users/456c74c7459347448a08d2856fdf714b => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:28.809128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:29.007935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910ba75-632f-424d-aaef-f5ad8703883d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:29.052572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:29.053251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:29.053551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:29.053797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:29.073429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:29.145299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:29.146817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:29.276600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 142/292] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:29.287209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:29.479453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1ad39-c414-4d2f-93c6-92e3784b565a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:29.509353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:29.538760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a527b-5745-4eb6-a055-642d098d0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:29.539356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a527b-5745-4eb6-a055-642d098d0b1f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:29.539643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a527b-5745-4eb6-a055-642d098d0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:29.539920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a527b-5745-4eb6-a055-642d098d0b1f None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:29.541192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a527b-5745-4eb6-a055-642d098d0b1f None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:29.542724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a527b-5745-4eb6-a055-642d098d0b1f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:29.545643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 151/293] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:29 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:29.554576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:29.638536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:29.722266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1062a37-e33e-4725-94d5-c1d3d9150775 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:29.770734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:29.771382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:29.771667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:29.771946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:29.772767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=456c74c7459347448a08d2856fdf714b) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:29.773126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:29.774641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01889041-c517-41d6-8936-1b1ce1077a5e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:29.782046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 143/294] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:29 2022] GET /identity/v3/role_assignments?user.id=456c74c7459347448a08d2856fdf714b&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:30.408243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616b342-408d-416b-bcb5-c083d2dcc755 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.408755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616b342-408d-416b-bcb5-c083d2dcc755 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.408854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616b342-408d-416b-bcb5-c083d2dcc755 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.410239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 152/295] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:30.416255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3edb86-3bd1-4cf4-bc2c-052c1c3505e9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.416787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3edb86-3bd1-4cf4-bc2c-052c1c3505e9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.416787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3edb86-3bd1-4cf4-bc2c-052c1c3505e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.437411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5d3edb86-3bd1-4cf4-bc2c-052c1c3505e9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:30.438657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d3edb86-3bd1-4cf4-bc2c-052c1c3505e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:30.509636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56589a02-d09f-4e3b-a931-65b89f598fc3 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.510336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56589a02-d09f-4e3b-a931-65b89f598fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.510811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56589a02-d09f-4e3b-a931-65b89f598fc3 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.512476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 153/296] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:30.521478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647239e8-422c-48ec-94b0-5d34ba5a1123 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.522089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647239e8-422c-48ec-94b0-5d34ba5a1123 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.522588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647239e8-422c-48ec-94b0-5d34ba5a1123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.524419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 144/297] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:30.543498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0c50-8281-4113-87df-24a618b25d0f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.543777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0c50-8281-4113-87df-24a618b25d0f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.544215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b0c50-8281-4113-87df-24a618b25d0f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.547390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 145/298] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:30.548224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b0c50-8281-4113-87df-24a618b25d0f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:30.555162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-647239e8-422c-48ec-94b0-5d34ba5a1123 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:30.556627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-647239e8-422c-48ec-94b0-5d34ba5a1123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:30.599144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:30.637031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 154/299] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:30.637031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.637031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.637383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.659336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:30.660696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:30.683475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b0c50-8281-4113-87df-24a618b25d0f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:30.730318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:30.731649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.732022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.732472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.734015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:30.736598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:30.738599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 146/300] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:30.739544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:30.745321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 155/301] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:30.746447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:30.806812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:30.865960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b394d-4c7d-4c16-b892-a940c5521b18 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:30.876119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:30.900367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d0950-9d23-415b-8c6c-db6c13062b0e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:30.916227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:30.916798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.917126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.917387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.930909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:30.937028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:30.938579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 147/302] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:30.939380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:30.958686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:30.959440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:30.959721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:30.960026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] PATH_INFO: `/v3/groups` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:30.960812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:30.962388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:30.982540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 156/303] 10.222.0.48 () {60 vars in 1134 bytes} [Tue Jul 26 16:57:30 2022] POST /identity/v3/groups => generated 229 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:57:30.983584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:31.149344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdfe4b-6de4-40e4-a257-f31f6457f7de None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.151817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b8b0d-4b1c-406b-b42f-c48dc83a5839 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.157271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:31.209840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.211744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.212750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.214065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.216148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:31.217205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:31.221393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:31.228327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.229033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.229321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.229597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.240799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 157/304] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:31.250180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:31.251223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:31.255623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:31.256633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:31.258792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 148/305] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:31.269040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:31.311261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:31.449139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e25fd-1374-47ee-9c84-6a8907d74abc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.473625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20ac56-011f-4f15-b81e-49bccdb1c87e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.516579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.517810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.518292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.518747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.518995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.519414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.519692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.519962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] PATH_INFO: `/v3/users/fc71e8a07dd3420da5dee4665c26046a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.520532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:31.521077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:31.522660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:31.536462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] RBAC: Authorizing `identity:get_user(user_id=fc71e8a07dd3420da5dee4665c26046a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:31.538655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:31.540724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 158/306] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:31 2022] GET /identity/v3/users/fc71e8a07dd3420da5dee4665c26046a => generated 293 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:31.542536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 149/307] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:31.549440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:31.551054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:31.579239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:31.677886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:31.686374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62917a-5473-4de8-b502-9800f4bc1ce4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.694173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef65dfa-4860-46be-a2a8-df46f8379d9a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.733998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.734639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.734920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.735190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.736012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] RBAC: Adding query filter params (role.id=70791be9532841bf9ab672f441e5b4b6, user.id=fc71e8a07dd3420da5dee4665c26046a, scope.system=all) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:31.736320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:31.736589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5858a-ae3e-4616-a8e1-52dffcc6d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.737512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5858a-ae3e-4616-a8e1-52dffcc6d358 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.737961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:31.738200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5858a-ae3e-4616-a8e1-52dffcc6d358 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.738659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5858a-ae3e-4616-a8e1-52dffcc6d358 None admin] PATH_INFO: `/v3/users/fa729a22c70840b39b6473f22cbbe635` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.740178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5858a-ae3e-4616-a8e1-52dffcc6d358 None admin] RBAC: Authorizing `identity:get_user(user_id=fa729a22c70840b39b6473f22cbbe635)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:31.741916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5858a-ae3e-4616-a8e1-52dffcc6d358 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:31.743914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 150/308] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:31 2022] GET /identity/v3/users/fa729a22c70840b39b6473f22cbbe635 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:31.749882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 159/309] 10.222.0.48 () {56 vars in 1308 bytes} [Tue Jul 26 16:57:31 2022] GET /identity/v3/role_assignments?user.id=fc71e8a07dd3420da5dee4665c26046a&scope.system=all&role.id=70791be9532841bf9ab672f441e5b4b6 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:31.751746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:31.859630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:31.875945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8278f1-62fe-45e8-9532-de29d0bb892c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:31.900331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97e045-6c00-481a-92c6-ca6fccf192a4 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.900822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97e045-6c00-481a-92c6-ca6fccf192a4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.901332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97e045-6c00-481a-92c6-ca6fccf192a4 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.905089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 151/310] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:31.910963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.911359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.911505 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.934630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:31.936102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:31.936835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:31.937160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:31.937438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] PATH_INFO: `/v3/system/users/fa729a22c70840b39b6473f22cbbe635/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:31.938030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:31.939221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fa729a22c70840b39b6473f22cbbe635, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:31.940717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:31.965544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 160/311] 10.222.0.48 () {58 vars in 1334 bytes} [Tue Jul 26 16:57:31 2022] PUT /identity/v3/system/users/fa729a22c70840b39b6473f22cbbe635/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:32.055140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 152/312] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:32.079531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:32.208695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cdc8a-eb55-46fa-9d6b-276b1e3b29f0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:32.251810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:32.252483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:32.252818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:32.253131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:32.266512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:32.272313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:32.273309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:32.274094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:32.278593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 161/313] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:32.282156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:32.431577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:32.454738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747909e-1c3e-475b-9143-e9ebce000e0e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:32.520517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:32.521717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:32.522188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:32.522679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:32.524012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:32.524541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:32.527238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:32.546681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 153/314] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:32.552169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:32.578852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:32.708001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12217-7042-44b1-b3ff-c6e5ad8542e1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:32.751351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:32.752376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:32.752842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:32.753328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] PATH_INFO: `/v3/users/7628d25ec3c84c39811980a3d4cc1c48` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:32.754866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] RBAC: Authorizing `identity:get_user(user_id=7628d25ec3c84c39811980a3d4cc1c48)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:32.756599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:32.758921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 162/315] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:32 2022] GET /identity/v3/users/7628d25ec3c84c39811980a3d4cc1c48 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:32.768872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:32.901787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983e292-cfb6-48e5-b8c4-80d394b72f24 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:32.941521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:32.942162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:32.942488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:32.942764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:32.964755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:33.033475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:33.034976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:33.158435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 154/316] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:33.164101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:33.268185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:33.302245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b76bb-78fa-4e3c-b5bc-f637a427cd40 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:33.370158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e7243-8310-4ae8-9401-57894fd36dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:33.371453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e7243-8310-4ae8-9401-57894fd36dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:33.372084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e7243-8310-4ae8-9401-57894fd36dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:33.373261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e7243-8310-4ae8-9401-57894fd36dd6 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:33.375609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e7243-8310-4ae8-9401-57894fd36dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:33.377848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e7243-8310-4ae8-9401-57894fd36dd6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:33.380130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 163/317] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:33 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:33.389548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:33.399205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08de3b-701e-49c7-851e-adca54d562f8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:33.399827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08de3b-701e-49c7-851e-adca54d562f8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:33.400306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08de3b-701e-49c7-851e-adca54d562f8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:33.401989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 164/318] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:33.411963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:33.412216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:33.412553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:33.433175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:33.434417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:33.485560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:33.521434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 165/319] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:33.536096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:33.578090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9306fe-799e-4545-8009-81e6cb6f4663 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:33.621060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:33.621976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:33.622407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:33.623402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:33.624363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] RBAC: Adding query filter params (role.id=70791be9532841bf9ab672f441e5b4b6, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0, user.id=7628d25ec3c84c39811980a3d4cc1c48) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:33.624948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:33.626607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:33.628789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:33.635088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 155/320] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:57:33 2022] GET /identity/v3/role_assignments?user.id=7628d25ec3c84c39811980a3d4cc1c48&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=70791be9532841bf9ab672f441e5b4b6 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:33.655454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bc35c-a40f-41e3-9765-2004654336bd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:33.697153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8bfba-4468-48d2-80f1-c672ef788607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:33.698160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8bfba-4468-48d2-80f1-c672ef788607 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:33.698595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8bfba-4468-48d2-80f1-c672ef788607 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:33.699067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8bfba-4468-48d2-80f1-c672ef788607 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:33.700581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8bfba-4468-48d2-80f1-c672ef788607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:33.702908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8bfba-4468-48d2-80f1-c672ef788607 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:33.705157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 166/321] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:33.710666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:33.837923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bef802-0204-4c3f-b996-e3589c2648d5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:33.855649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:33.882912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1000-f36b-4288-9ac8-707213387f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:33.883562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1000-f36b-4288-9ac8-707213387f5e None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:33.883847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1000-f36b-4288-9ac8-707213387f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:33.884119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1000-f36b-4288-9ac8-707213387f5e None admin] PATH_INFO: `/v3/groups` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:33.884819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d1000-f36b-4288-9ac8-707213387f5e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:33.886359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d1000-f36b-4288-9ac8-707213387f5e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:33.906034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 156/322] 10.222.0.48 () {60 vars in 1134 bytes} [Tue Jul 26 16:57:33 2022] POST /identity/v3/groups => generated 226 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:57:34.384410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09930e-a6e7-4f4d-ab78-2e16a2aebee8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:34.385190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09930e-a6e7-4f4d-ab78-2e16a2aebee8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:34.385695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09930e-a6e7-4f4d-ab78-2e16a2aebee8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:34.387636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 167/323] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:34.393941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfdee4f-7d90-468b-a01f-3617433a7226 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:34.394568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfdee4f-7d90-468b-a01f-3617433a7226 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:34.395085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfdee4f-7d90-468b-a01f-3617433a7226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:34.431198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fcfdee4f-7d90-468b-a01f-3617433a7226 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:34.432468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fcfdee4f-7d90-468b-a01f-3617433a7226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:34.541392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 157/324] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:34.567519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09930e-a6e7-4f4d-ab78-2e16a2aebee8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:34.577449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8e90d-2ae5-4c45-a62e-ae10d5f6db8f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:34.577842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8e90d-2ae5-4c45-a62e-ae10d5f6db8f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:34.578017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8e90d-2ae5-4c45-a62e-ae10d5f6db8f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:34.579874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 158/325] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:34.585811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:34.586232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:34.586377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:34.607234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:34.608940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:34.695742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 159/326] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:34.723793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09930e-a6e7-4f4d-ab78-2e16a2aebee8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:34.725484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:34.766993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:34.798215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:34.799141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:34.799567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:34.799982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:34.818437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:34.823841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:34.824899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:34.829180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:34.831894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 168/327] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:34.841006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:34.866839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:34.915116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0dca3-b5a0-412e-98c2-e5313d3e6f67 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:35.011490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:35.012508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:35.015242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:35.015695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:35.027030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b0354-4a49-4636-ad71-4eab79ad0447 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:35.045058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:35.051979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:35.052932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:35.054150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:35.057662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 160/328] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:35.064115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:35.098648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:35.100033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:35.100549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:35.101120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:35.103341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:35.103912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:35.107208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:35.119696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 169/329] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:35.129227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:35.248893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:35.269399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644cdc8-c1d3-4d35-bab1-9cfcc813e35c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:35.272174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:35.353224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:35.354939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:35.355719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:35.356504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:35.359475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:35.362506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:35.366134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:35.388108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 161/330] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:35.393551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039a108-1eea-44e5-8e43-166a5c65fd67 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:35.398632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:35.518962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:35.519816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:35.520041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:35.520407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] PATH_INFO: `/v3/users/fc71e8a07dd3420da5dee4665c26046a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:35.523472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] RBAC: Authorizing `identity:get_user(user_id=fc71e8a07dd3420da5dee4665c26046a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:35.525581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:35.529957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 170/331] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:35 2022] GET /identity/v3/users/fc71e8a07dd3420da5dee4665c26046a => generated 293 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:35.535033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:35.608541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cf6c9-1a10-4158-be76-48a2c95fd3e0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:35.692200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:35.693541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:35.694053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:35.694498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] PATH_INFO: `/v3/users/fa729a22c70840b39b6473f22cbbe635` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:35.702635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] RBAC: Authorizing `identity:get_user(user_id=fa729a22c70840b39b6473f22cbbe635)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:35.705679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:35.709583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 162/332] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:35 2022] GET /identity/v3/users/fa729a22c70840b39b6473f22cbbe635 => generated 293 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:35.719378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8402f7-ffc1-48d0-8899-1a34c4190242 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:35.719778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:35.770314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd04bb-0905-4f6c-b7e1-3f8d34752171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:35.771680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd04bb-0905-4f6c-b7e1-3f8d34752171 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:35.772174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd04bb-0905-4f6c-b7e1-3f8d34752171 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:35.772663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd04bb-0905-4f6c-b7e1-3f8d34752171 None admin] PATH_INFO: `/v3/system/users/fc71e8a07dd3420da5dee4665c26046a/roles/70791be9532841bf9ab672f441e5b4b6` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:35.774610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd04bb-0905-4f6c-b7e1-3f8d34752171 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fc71e8a07dd3420da5dee4665c26046a, role_id=70791be9532841bf9ab672f441e5b4b6)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:35.776442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd04bb-0905-4f6c-b7e1-3f8d34752171 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:35.781686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:35.790864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 171/333] 10.222.0.48 () {58 vars in 1334 bytes} [Tue Jul 26 16:57:35 2022] PUT /identity/v3/system/users/fc71e8a07dd3420da5dee4665c26046a/roles/70791be9532841bf9ab672f441e5b4b6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:57:35.808564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:35.975196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a1b81-5999-47f5-8b08-7b7d698da3f6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:36.023515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:36.024871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:36.025201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:36.025481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:36.026321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] RBAC: Adding query filter params (role.id=94d6523ed4be482abd3484d9adce8e0b, user.id=fa729a22c70840b39b6473f22cbbe635, scope.system=all) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:36.026631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:36.028185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a85a-2a83-4558-b433-41aa339c91e1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:36.055824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 163/334] 10.222.0.48 () {56 vars in 1308 bytes} [Tue Jul 26 16:57:35 2022] GET /identity/v3/role_assignments?user.id=fa729a22c70840b39b6473f22cbbe635&scope.system=all&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 510 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:37.365947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4359e4-a4d3-47e1-8a26-d031e1c4b7e7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:37.366758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4359e4-a4d3-47e1-8a26-d031e1c4b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:37.366758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4359e4-a4d3-47e1-8a26-d031e1c4b7e7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:37.368284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 172/335] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:37.382893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:37.383576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:37.383576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:37.411120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:37.412398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:37.555200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 164/336] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:37.598145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4359e4-a4d3-47e1-8a26-d031e1c4b7e7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:37.743003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4359e4-a4d3-47e1-8a26-d031e1c4b7e7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:37.803470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:37.804603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:37.805133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:37.805632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:37.835409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:37.850306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:37.851276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:37.852369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:37.853512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 173/337] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:37.860933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:37.891975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:38.119943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba1998-2241-4882-ba64-c7c1ce26ed78 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:38.211662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:38.212797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:38.213330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:38.213826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:38.215150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:38.215698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:38.218337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf27279-0d7e-4b52-9d7d-bf0bbd03b676 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:38.240599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 165/338] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:38.246007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:38.396393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:38.454406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15687c5-d243-4508-8704-ff39d26e581b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:38.454911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15687c5-d243-4508-8704-ff39d26e581b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:38.455101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15687c5-d243-4508-8704-ff39d26e581b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:38.457015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 166/339] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:38.472775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:38.473303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:38.473806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:38.505641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:38.507270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:38.512990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abd453-e719-4bd3-b005-51448458cb4b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:38.610364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:38.611396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:38.611913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:38.612394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:38.693426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 167/340] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:38.775796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:38.782622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:38.784516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:38.793460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:38.952375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:39.040891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 174/341] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:39.048054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:39.105893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cec28-6d3c-49f8-bcb9-f06dddd085fd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:39.246300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:39.247346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.248045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.248544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.293235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:39.303327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:39.304308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:39.305221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:39.306468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 168/342] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 531 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:39.315220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:39.363739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae0787-e19d-4b8f-8094-3ed3b831830c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:39.457813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d571868-c74f-402f-80f3-324b846d876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:39.459460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571868-c74f-402f-80f3-324b846d876a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.459975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571868-c74f-402f-80f3-324b846d876a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.460520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d571868-c74f-402f-80f3-324b846d876a None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.462228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571868-c74f-402f-80f3-324b846d876a None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:39.464123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d571868-c74f-402f-80f3-324b846d876a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:39.466346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 175/343] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:39 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:39.468033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211f9e-75b7-4dfb-ad73-b0cf2c9d8c03 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.468551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211f9e-75b7-4dfb-ad73-b0cf2c9d8c03 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.469079 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211f9e-75b7-4dfb-ad73-b0cf2c9d8c03 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.470694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 176/344] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:39.474299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:39.477965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44211f9e-75b7-4dfb-ad73-b0cf2c9d8c03 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:39.545003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:39.580638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f418505-29d3-4636-95f0-35a0b96c5dcb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:39.649046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:39.649285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.649670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.650180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.651587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:39.652143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:39.655675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac668b4-0eb7-4668-8fac-e012e1362a9b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:39.674988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 169/345] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:39.676847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d68669-c5d6-42b4-86db-e453348b4ca7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.677492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d68669-c5d6-42b4-86db-e453348b4ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.677983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d68669-c5d6-42b4-86db-e453348b4ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.702822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b8d68669-c5d6-42b4-86db-e453348b4ca7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:39.704447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b8d68669-c5d6-42b4-86db-e453348b4ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:39.746523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44211f9e-75b7-4dfb-ad73-b0cf2c9d8c03 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:39.791339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 170/346] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:39.793361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf08e-dd89-4c15-9b39-d7155fcf996c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.793958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf08e-dd89-4c15-9b39-d7155fcf996c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.794586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaf08e-dd89-4c15-9b39-d7155fcf996c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.796862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 171/347] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:39.798043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf08e-dd89-4c15-9b39-d7155fcf996c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:39.876078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10441efe-9002-4928-b756-082953c03eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:39.877275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441efe-9002-4928-b756-082953c03eee None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.878081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441efe-9002-4928-b756-082953c03eee None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.878414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10441efe-9002-4928-b756-082953c03eee None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.878884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:39.893003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441efe-9002-4928-b756-082953c03eee None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:39.895880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10441efe-9002-4928-b756-082953c03eee None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:39.899401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 177/348] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:39 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:39.902254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ece05-1bbf-4335-8f85-459b6dca81d7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:39.903135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ece05-1bbf-4335-8f85-459b6dca81d7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:39.903615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ece05-1bbf-4335-8f85-459b6dca81d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:39.940120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f09ece05-1bbf-4335-8f85-459b6dca81d7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:39.943073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f09ece05-1bbf-4335-8f85-459b6dca81d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:39.968392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaf08e-dd89-4c15-9b39-d7155fcf996c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:40.051166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be98df-2a0f-4446-9fd5-7ae1a3d73e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:40.052030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be98df-2a0f-4446-9fd5-7ae1a3d73e1a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.052467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be98df-2a0f-4446-9fd5-7ae1a3d73e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.053052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be98df-2a0f-4446-9fd5-7ae1a3d73e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.068123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 178/349] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:40.069686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c909b6-b411-4d00-877f-2b9613da4670 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.070034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c909b6-b411-4d00-877f-2b9613da4670 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.070407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c909b6-b411-4d00-877f-2b9613da4670 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.072925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 179/350] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:40.073790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c909b6-b411-4d00-877f-2b9613da4670 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:40.114500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:40.195587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be98df-2a0f-4446-9fd5-7ae1a3d73e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:40.198442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be98df-2a0f-4446-9fd5-7ae1a3d73e1a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:40.263864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c909b6-b411-4d00-877f-2b9613da4670 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:40.300846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:40.335804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:40.336898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.337223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.337506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.356854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:40.369038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:40.370846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 180/351] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:40.372334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:40.387036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 172/352] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:40.388928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.389247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.389527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.417372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:40.419875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:40.528237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:40.539367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 173/353] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:40.541764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:40.552748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3d382-8920-4a34-b224-3dae63a5d4a9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:40.597272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:40.597839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.598168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.598450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.599293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=5e8bf6cf3bb44be2a505004317684749) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:40.599604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:40.601259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e232333-2281-4244-9a7f-894ae1c5e05a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:40.608544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 181/354] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:40 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:40.610145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3daa6-fe0e-443e-8db0-a1b65df81907 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.610267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3daa6-fe0e-443e-8db0-a1b65df81907 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.610493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3daa6-fe0e-443e-8db0-a1b65df81907 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.612147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 182/355] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:40.612700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3daa6-fe0e-443e-8db0-a1b65df81907 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:40.704080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:40.736830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42108f21-c51b-481d-bce4-73a9564f37ee None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:40.803692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:40.804491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.804853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.805225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.817699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3daa6-fe0e-443e-8db0-a1b65df81907 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:40.824896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:40.834729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:40.835704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 16:57:40.836503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:40.837673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 174/356] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:40.839218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:40.881848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:40.883250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:40.883727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:40.884193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:40.886708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:40.886952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:40.890662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:40.908258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 183/357] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:40.910090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:40.934651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:41.031013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db2b10-0b66-4c60-8a2a-c01d1fc97374 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:41.104341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8211103-f349-43fc-a2b9-2556321fc8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:41.104719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211103-f349-43fc-a2b9-2556321fc8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:41.105247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211103-f349-43fc-a2b9-2556321fc8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:41.106026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211103-f349-43fc-a2b9-2556321fc8b4 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:41.110291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211103-f349-43fc-a2b9-2556321fc8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:41.112184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8211103-f349-43fc-a2b9-2556321fc8b4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:41.115860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 175/358] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:40 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:41.126717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:41.127204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:41.127693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:41.134250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32a48b-d52a-49ea-a975-2422348e67af None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:41.160441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:41.161731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:41.171641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:41.207557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:41.208905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:41.209454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:41.209912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:41.229423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:41.238750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:41.240345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:41.241871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 184/359] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:41.242596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:41.296081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 176/360] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:41.297846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:41.377305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:41.461598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba74ed-fd1c-4ba1-884d-f40697d7142d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:41.535134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6e9ed-1c01-4c97-acd1-96ffa520f112 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:41.539523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:41.540535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:41.542413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:41.542992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:41.544330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:41.545420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:41.548063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:41.561179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 185/361] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:41.562386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:41.607571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:41.608678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:41.609379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:41.610105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:41.636224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:41.714673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969a05e-3e76-4e0f-a4b8-7bcd476171a4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:41.770943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:41.773157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:41.781107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:41.794308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:41.796250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:41.796518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:41.797934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:41.798383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:41.799890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:41.802311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 186/362] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:41 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:41.803297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:41.955355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 177/363] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:41.960938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:41.982363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edf13d-37e5-4401-95ef-341078243e4e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.020733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:42.027590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.028312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.028602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.028934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.029819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:42.030252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:42.031720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:42.048107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 187/364] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:42.048107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:42.108172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb6302-5a73-484d-82f2-176e17e9f8df None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.120628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:42.165984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.167108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.167781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.168285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.173326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334de207-4eb3-4fba-ac2f-7c82290c54ce None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.189142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:42.199536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:42.201499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:42.202497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:42.203561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 178/365] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:42.204877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:42.223519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.224188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.224468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.224740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] PATH_INFO: `/v3/users/fc71e8a07dd3420da5dee4665c26046a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.226085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] RBAC: Authorizing `identity:get_user(user_id=fc71e8a07dd3420da5dee4665c26046a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:42.227858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:42.229639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 188/366] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:57:42 2022] GET /identity/v3/users/fc71e8a07dd3420da5dee4665c26046a => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:42.235651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:42.359691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292cf1f-3dfc-4bc3-bc6a-771434446922 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.361777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:42.394466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063f40b-8e22-4ac4-90c8-89a6883578dc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.405546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.406122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.406403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.406681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.407528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=5e8bf6cf3bb44be2a505004317684749) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:42.407843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:42.409349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:42.425024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 179/367] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:42 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=55b0174dc86246af94379fcc1ef68040 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:42.425024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:42.447160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:42.458433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.459107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.459391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.459669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.461437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:42.463735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:42.466877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 189/368] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:42 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:42.468171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:42.674394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459adf-5133-42bb-a87e-62e68ec48804 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.688643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ba1d-c2c6-4d02-b7e5-a5fa7927ae8b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:42.729803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.730807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.731300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.731717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.771115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:42.772544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:42.773069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:42.773539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:42.775150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:42.775689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:42.779752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:42.805081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:42.805335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 190/369] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:42.805335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:42.860721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:42.863935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:42.937307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:43.030094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adde99-7dcf-47a0-b973-80148b08482d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:43.089558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 180/370] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:43.091561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:43.103208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:43.104360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:43.104834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:43.105360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:43.106842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] RBAC: Adding query filter params (role.id=70791be9532841bf9ab672f441e5b4b6, user.id=fc71e8a07dd3420da5dee4665c26046a, scope.system=all) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:43.109002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:43.111370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:43.137004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 191/371] 10.222.0.48 () {56 vars in 1308 bytes} [Tue Jul 26 16:57:42 2022] GET /identity/v3/role_assignments?user.id=fc71e8a07dd3420da5dee4665c26046a&scope.system=all&role.id=70791be9532841bf9ab672f441e5b4b6 => generated 510 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:43.138989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:43.245021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:43.320064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9500f-3fe7-428d-a2cd-46acedc77259 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:43.431364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:43.432132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:43.432435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:43.432743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:43.434555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:43.436390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:43.440715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 181/372] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:43 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:43.444932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:43.479243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf1a9-7118-4c1c-a92b-32a378b8cd9e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:43.511427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:43.544422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd163c-d59e-498d-ae69-5e44784d1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:43.545810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd163c-d59e-498d-ae69-5e44784d1026 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:43.546087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd163c-d59e-498d-ae69-5e44784d1026 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:43.546356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd163c-d59e-498d-ae69-5e44784d1026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:43.656259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7679a9-669e-4d42-acc0-2757827b9e8b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:43.710823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd163c-d59e-498d-ae69-5e44784d1026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:43.713762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd163c-d59e-498d-ae69-5e44784d1026 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:43.714395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:43.715845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:43.717017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:43.717731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:43.720348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:43.724208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:43.726983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 182/373] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:43 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:43.728587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:43.733554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:43.916832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 192/374] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:43.918407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba474b-f575-4bf6-ab81-2f154c072f03 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:43.918710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba474b-f575-4bf6-ab81-2f154c072f03 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:43.918990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba474b-f575-4bf6-ab81-2f154c072f03 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:43.921870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 193/375] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:43.922787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba474b-f575-4bf6-ab81-2f154c072f03 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:43.939860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661b87f-dcd1-4d42-9411-dbedc87c59ea None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:43.949367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:44.028304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.029885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.030301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.030672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.031933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:44.032345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:44.034644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:44.044384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 183/376] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:43 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 260 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:44.048297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:44.094638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba474b-f575-4bf6-ab81-2f154c072f03 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:44.151876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:44.168302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558dbc-8bf6-49b7-8db8-acc7a5947fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.169293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558dbc-8bf6-49b7-8db8-acc7a5947fac None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.170511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558dbc-8bf6-49b7-8db8-acc7a5947fac None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.170843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558dbc-8bf6-49b7-8db8-acc7a5947fac None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.173188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf558dbc-8bf6-49b7-8db8-acc7a5947fac None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:44.175909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf558dbc-8bf6-49b7-8db8-acc7a5947fac None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:44.180524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 194/377] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:43 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:44.182969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.183410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.183897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.207350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4abb07-1500-45d6-96de-cabb965a2e8f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:44.230310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:44.232671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:44.297592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.299723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.300685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.301617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] PATH_INFO: `/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.304201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] RBAC: Authorizing `identity:get_project(project_id=f28d3ad4783445e5a27ff264d5fb4f43)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:44.308996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:44.313198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 184/378] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:44 2022] GET /identity/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:44.315157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:44.383310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 195/379] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:44.383310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:44.408658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:44.541572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a31fbe-85df-448c-b293-9484df4d15a7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:44.571742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fe61b-16fa-4deb-8045-ef1527cfb308 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:44.613030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.614478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.614947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.615405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] PATH_INFO: `/v3/groups/e48ee57702f642b984bd193983076f76` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.623872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] RBAC: Authorizing `identity:get_group(group_id=e48ee57702f642b984bd193983076f76)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:44.626056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:44.628504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 196/380] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:44 2022] GET /identity/v3/groups/e48ee57702f642b984bd193983076f76 => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:44.630084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:44.646095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.647340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.647925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.648507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.650236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:44.650910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:44.653833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:44.657441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:44.666237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 185/381] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:44 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=55b0174dc86246af94379fcc1ef68040 => generated 260 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:44.667810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:44.819958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:44.865104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e400b8-181f-4529-8460-57d4b312f0d7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:44.882878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b704b6-b456-4d41-8b5a-c32e98fe8797 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:44.941243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.941909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.942193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.942471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.943320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] RBAC: Adding query filter params (group.id=e48ee57702f642b984bd193983076f76, role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=f28d3ad4783445e5a27ff264d5fb4f43) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:44.943632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:44.945215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:44.950433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:44.951160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:44.951443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:44.951715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:44.964142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 186/382] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:44 2022] GET /identity/v3/role_assignments?group.id=e48ee57702f642b984bd193983076f76&scope.project.id=f28d3ad4783445e5a27ff264d5fb4f43&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 260 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:44.986990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:44.992611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:44.994957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:44.998456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 197/383] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:45.009140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:45.090180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:45.143751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff333365-f893-4e3d-928e-a4a7d1b7ed4e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:45.192878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:45.193565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:45.193850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:45.194125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:45.194903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:45.195210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:45.196678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:45.210623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 187/384] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:45.219575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:45.347140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf3968-0cbb-4ae6-9786-a5c87a6ddcf8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:45.399585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20334e54-67d3-409b-b816-21a955f7cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:45.400266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20334e54-67d3-409b-b816-21a955f7cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:45.400546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20334e54-67d3-409b-b816-21a955f7cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:45.400824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20334e54-67d3-409b-b816-21a955f7cfa5 None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:45.402481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20334e54-67d3-409b-b816-21a955f7cfa5 None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:45.404081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20334e54-67d3-409b-b816-21a955f7cfa5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:45.414315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 198/385] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:45 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:45.422014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:45.544305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3127b-53a0-4d1a-a9be-864c9d091794 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:45.545249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3127b-53a0-4d1a-a9be-864c9d091794 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:45.545249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3127b-53a0-4d1a-a9be-864c9d091794 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:45.547704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 199/386] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:45.556318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:45.556734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:45.556999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:45.573085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:45.581873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:45.583309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:45.638930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726ee72-81c4-45d2-be0d-2042e449c455 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:45.684826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:45.710828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:45.711499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:45.711775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:45.712042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:45.714536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:45.716158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:45.718516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 188/387] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:45 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:45.728783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:45.744694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 200/388] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:45.814429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:45.929103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a66555-6896-49c7-8c9f-af4c9551a9db None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:45.986484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:45.987405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:45.987736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:45.988069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:45.990776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bf6cf3bb44be2a505004317684749, group_id=3d06338f42b049dc9a6d0d4b16c1ac79, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:45.992427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:46.011952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 189/389] 10.222.0.48 () {58 vars in 1442 bytes} [Tue Jul 26 16:57:45 2022] PUT /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:46.071204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206597b6-52cf-4093-bc75-ba3c0ba6af4f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:46.131840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:46.132808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:46.133268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:46.133683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:46.142418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:46.158032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:46.163268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:46.164264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:46.165117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 201/390] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:46.175528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:46.293873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:46.360006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa34345-1a8b-4a58-ab5e-8619eae71d76 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:46.407491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:46.408235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:46.408572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:46.408900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:46.409805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:46.410159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:46.411778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:46.424481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 190/391] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:46.432742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:46.626504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4efc7-2c6f-4d8f-9e08-b3ff75ee7be7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:46.678775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:46.679547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:46.679828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:46.680114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:46.681552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:46.683070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:46.685943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 202/392] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:46 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:46.694095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:46.891482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da318e-c378-420b-9678-f9258e0633d8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:46.917521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:46.947979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bdb55-cb28-48a2-9f8f-cde768b941e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:46.949014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdb55-cb28-48a2-9f8f-cde768b941e0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:46.949360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdb55-cb28-48a2-9f8f-cde768b941e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:46.949650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdb55-cb28-48a2-9f8f-cde768b941e0 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:46.952196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bdb55-cb28-48a2-9f8f-cde768b941e0 None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:46.954306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bdb55-cb28-48a2-9f8f-cde768b941e0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:46.957084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 191/393] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:46 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:46.965234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:47.077422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:47.133759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085e413-736d-4f51-af74-ba9cdf4a0569 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:47.186941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c5eaf-8de5-4177-989f-01eb30421142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:47.187669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c5eaf-8de5-4177-989f-01eb30421142 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:47.188023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c5eaf-8de5-4177-989f-01eb30421142 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:47.188368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c5eaf-8de5-4177-989f-01eb30421142 None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:47.191516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c5eaf-8de5-4177-989f-01eb30421142 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e8bf6cf3bb44be2a505004317684749, group_id=3d06338f42b049dc9a6d0d4b16c1ac79, role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:47.193589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c5eaf-8de5-4177-989f-01eb30421142 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:47.215823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 203/394] 10.222.0.48 () {58 vars in 1442 bytes} [Tue Jul 26 16:57:46 2022] PUT /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/55b0174dc86246af94379fcc1ef68040 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:47.472756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30661c-9e4f-4878-9b3b-9598f3950d40 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:47.473051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30661c-9e4f-4878-9b3b-9598f3950d40 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:47.473407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30661c-9e4f-4878-9b3b-9598f3950d40 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:47.475131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 192/395] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:47.482689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:47.483649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:47.484425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:47.524930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:47.526835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:47.642042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 204/396] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:47.669296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30661c-9e4f-4878-9b3b-9598f3950d40 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:47.855168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30661c-9e4f-4878-9b3b-9598f3950d40 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:47.914977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:47.915712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:47.916069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:47.916347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:47.936910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:47.942595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:47.944405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:47.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 193/397] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:57:47.955337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:48.122842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:48.211018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c819c1-8601-4cd4-a6b8-561061b62433 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:48.268552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:48.270123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.270507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.270902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.277217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:48.277626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:48.279238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ce3a9-dd43-4caf-a213-5243e8261b17 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:48.292130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 205/398] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:48.298901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:48.304921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e83fdb-0452-478d-9a69-0bf5433f5304 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.305546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e83fdb-0452-478d-9a69-0bf5433f5304 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.306002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e83fdb-0452-478d-9a69-0bf5433f5304 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.307975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 206/399] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:48.314741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.315288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.315710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.335668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:48.337268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:48.418185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:48.493450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 207/400] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:48.536627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313b39c-46dd-48c3-9a9b-444d4b4f5471 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:48.545396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:48.625439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:48.625700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a116936-9533-4da6-962c-ac4ee55926cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:48.626663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116936-9533-4da6-962c-ac4ee55926cc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.627048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116936-9533-4da6-962c-ac4ee55926cc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.627884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116936-9533-4da6-962c-ac4ee55926cc None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.629831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a116936-9533-4da6-962c-ac4ee55926cc None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:48.632700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a116936-9533-4da6-962c-ac4ee55926cc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:48.637221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 194/401] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:48 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:48.639252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4105c-ae52-4f1e-9017-0e17eec814d0 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.639606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4105c-ae52-4f1e-9017-0e17eec814d0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.640096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4105c-ae52-4f1e-9017-0e17eec814d0 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.642631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 195/402] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:48.649537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4105c-ae52-4f1e-9017-0e17eec814d0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:48.702678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06851e48-b5dd-4232-a0f6-d69b5f2fd193 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:48.748420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687cb763-dce4-4b43-8685-16ccae210b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:48.749235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cb763-dce4-4b43-8685-16ccae210b31 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.749516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cb763-dce4-4b43-8685-16ccae210b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.749792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687cb763-dce4-4b43-8685-16ccae210b31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.765165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687cb763-dce4-4b43-8685-16ccae210b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-687cb763-dce4-4b43-8685-16ccae210b31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:48.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:48.774369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 208/403] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:48.777470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.790346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.790346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.804348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:48.805391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4105c-ae52-4f1e-9017-0e17eec814d0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:48.805885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:48.824172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:48.873703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:48.874752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:48.875046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:48.875373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:48.877861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:48.879699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:48.882873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 196/404] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:48 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:48.883985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:48.904664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 209/405] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:48.905713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:48.962945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:49.051945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1e095-cef7-4360-aa93-fb98515aa6dd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.063653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d99eb-4f70-4d06-a3c1-165d5adcf66d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.113865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:49.114655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:49.114938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:49.115211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:49.116055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:49.116357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:49.116597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:49.117353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:49.117677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:49.117941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:49.118178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:49.120687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] RBAC: Authorizing `identity:create_grant(project_id=20d464db83b0461b8a5f3a3fd94529c0, group_id=3d06338f42b049dc9a6d0d4b16c1ac79, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:49.122495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:49.133184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 197/406] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:49.134418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:49.149451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 210/407] 10.222.0.48 () {58 vars in 1442 bytes} [Tue Jul 26 16:57:48 2022] PUT /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:49.150894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:49.225058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:49.305254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:49.318746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1fdd0-d5e0-4250-8666-ae26d4a393ed None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.371959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:49.373518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:49.373881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:49.374223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:49.376144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:49.377935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:49.382152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 211/408] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:49 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:49.388390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:49.442094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9fed3d-c6b2-4d35-ab65-5f720c8d68e7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.502201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:49.502935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:49.503244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:49.503546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:49.528089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:49.543448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:49.544485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:49.546134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:49.549179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 198/409] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:49.554904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:49.576990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632f7d-bbdf-4200-9789-185a6dc79019 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.677386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:49.678722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:49.679216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:49.679689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:49.681491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:49.683722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:49.694474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 212/410] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:49 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:49.696136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:49.768926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:49.865385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:49.871610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa61fe4-fa65-4347-b675-839b670b0b20 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.956874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ab021-e261-49a0-8025-41bd588a1789 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:49.964145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:49.965280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:49.965762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:49.966277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:49.967679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:49.968242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:49.971009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbefedd-9894-46fd-a125-cacf4958eb12 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:49.996651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 199/411] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:50.001662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ea258-1d46-4013-add6-529fb166f1ae None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.003078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ea258-1d46-4013-add6-529fb166f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.004623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ea258-1d46-4013-add6-529fb166f1ae None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.009285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 200/412] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:50.011625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ea258-1d46-4013-add6-529fb166f1ae None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.015329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7936a0-9f37-4de5-a84a-f15e251de6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.016154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7936a0-9f37-4de5-a84a-f15e251de6b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.016536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7936a0-9f37-4de5-a84a-f15e251de6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.016818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7936a0-9f37-4de5-a84a-f15e251de6b7 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.021047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7936a0-9f37-4de5-a84a-f15e251de6b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=20d464db83b0461b8a5f3a3fd94529c0, group_id=3d06338f42b049dc9a6d0d4b16c1ac79, role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:50.021953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7936a0-9f37-4de5-a84a-f15e251de6b7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:50.040771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 213/413] 10.222.0.48 () {58 vars in 1442 bytes} [Tue Jul 26 16:57:49 2022] PUT /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0/groups/3d06338f42b049dc9a6d0d4b16c1ac79/roles/55b0174dc86246af94379fcc1ef68040 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:50.042775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49549717-909e-4c73-bf27-45fe04836bfa None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.043156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49549717-909e-4c73-bf27-45fe04836bfa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.043471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49549717-909e-4c73-bf27-45fe04836bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.072137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-49549717-909e-4c73-bf27-45fe04836bfa None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:50.074144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-49549717-909e-4c73-bf27-45fe04836bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:50.205931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ea258-1d46-4013-add6-529fb166f1ae None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:50.208450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 214/414] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:50.211019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb117-376c-4d6a-8fa8-2873feab88a6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.211530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb117-376c-4d6a-8fa8-2873feab88a6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.212025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb117-376c-4d6a-8fa8-2873feab88a6 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.215114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 215/415] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:50.223316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.223764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.224046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.251288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:50.252904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:50.268009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.269253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.269630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.269942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] PATH_INFO: `/v3/groups/e48ee57702f642b984bd193983076f76` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.272068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] RBAC: Authorizing `identity:get_group(group_id=e48ee57702f642b984bd193983076f76)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:50.273979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:50.280992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 201/416] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/groups/e48ee57702f642b984bd193983076f76 => generated 229 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:50.280992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.281317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:50.328470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:50.389865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 216/417] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:50.391485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.440424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e7643-859e-4e65-b917-c1f9b285243e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:50.491161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.492071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.492461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.492902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.508354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:50.516793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:50.518002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:50.519514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:50.520981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 202/418] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:50.520981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.597564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07e10a-ca7b-4bc5-ad43-edc62fdca1ec None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:50.652906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:50.663358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.664475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.664915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.665397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] PATH_INFO: `/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.667579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] RBAC: Authorizing `identity:get_project(project_id=f28d3ad4783445e5a27ff264d5fb4f43)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:50.670681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:50.670969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2e3e0-712c-40d2-9144-f0a652d7b4af None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:50.674120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 217/419] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43 => generated 344 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:50.675619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.718224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:50.725830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.726968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.727905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.728545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.744455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:50.752403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:50.753686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:50.754889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:50.765033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 203/420] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:50.766565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.873502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8cf06-5b28-4dfb-a6f1-d08420bb2cab None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:50.915038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a67fc-3baa-44fa-aabc-075bfbb14167 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:50.943106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.944260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.944770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.945511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.946940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:50.947509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:50.950572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:50.963907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:50.964739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:50.965055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:50.965333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] PATH_INFO: `/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43/groups/e48ee57702f642b984bd193983076f76/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:50.967137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 218/421] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:50.968414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] RBAC: Authorizing `identity:create_grant(project_id=f28d3ad4783445e5a27ff264d5fb4f43, group_id=e48ee57702f642b984bd193983076f76, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:50.968579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:50.969973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:50.997281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 204/422] 10.222.0.48 () {58 vars in 1442 bytes} [Tue Jul 26 16:57:50 2022] PUT /identity/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43/groups/e48ee57702f642b984bd193983076f76/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:57:51.001315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:51.041571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:51.151422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:51.194254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f259904-c63c-42b5-9bf0-e0ac14872b66 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:51.268701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8cd04-c7f2-49ab-9bc4-0da0e96da443 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:51.275340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:51.276504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:51.277035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:51.277543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:51.279066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:51.279685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:51.282502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:51.307133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 219/423] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:51.318544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:51.335618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:51.336620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:51.337159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:51.337635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:51.434003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:51.436793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:51.453709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:51.583332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 205/424] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:51.590713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:51.668096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8662cda-5f2c-4bcd-82a7-fb024eb8404e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:51.751258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f612dd7-525d-425a-91cf-469887e94fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:51.753727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f612dd7-525d-425a-91cf-469887e94fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:51.754509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f612dd7-525d-425a-91cf-469887e94fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:51.755198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f612dd7-525d-425a-91cf-469887e94fe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:51.784608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:51.866514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d773-c514-4b0a-872f-2b486e47b43e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:51.889904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:51.892681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f612dd7-525d-425a-91cf-469887e94fe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:51.895925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f612dd7-525d-425a-91cf-469887e94fe5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:51.945409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810053d-be94-4d75-8ede-4a596adba1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:51.947032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810053d-be94-4d75-8ede-4a596adba1fa None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:51.947727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810053d-be94-4d75-8ede-4a596adba1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:51.948412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810053d-be94-4d75-8ede-4a596adba1fa None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:51.951192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810053d-be94-4d75-8ede-4a596adba1fa None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:51.954417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810053d-be94-4d75-8ede-4a596adba1fa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:51.958721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 206/425] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:51 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:51.961471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7aaa8-76cc-4ae4-a1b1-3ff8515517fb None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:51.962204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7aaa8-76cc-4ae4-a1b1-3ff8515517fb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:51.962895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7aaa8-76cc-4ae4-a1b1-3ff8515517fb None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:51.973398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 207/426] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:51.975256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7aaa8-76cc-4ae4-a1b1-3ff8515517fb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:52.090747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 220/427] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:52.094572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:52.095449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:52.096186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:52.129511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:52.132393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:52.176130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7aaa8-76cc-4ae4-a1b1-3ff8515517fb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:52.212694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:52.245873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:52.247607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:52.248459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:52.249299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:52.251784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:52.254954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:52.259165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 208/428] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:51 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:52.261339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:52.277373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 221/429] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:52.280562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:52.356758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:52.472032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878972a2-7dfb-46b8-9b9c-825f32ff4bf9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:52.514740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704cf8e-8652-4c82-91b7-99459d586784 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:52.544161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:52.545652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:52.546068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:52.546568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] PATH_INFO: `/v3/projects/5e8bf6cf3bb44be2a505004317684749` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:52.548941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] RBAC: Authorizing `identity:get_project(project_id=5e8bf6cf3bb44be2a505004317684749)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:52.552113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:52.561887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 209/430] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:52 2022] GET /identity/v3/projects/5e8bf6cf3bb44be2a505004317684749 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:52.563189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:52.570637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:52.572011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:52.572630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:52.573249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:52.574582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=5e8bf6cf3bb44be2a505004317684749) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:52.575273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:52.578807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:52.591065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 222/431] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:52 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 611 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:52.593019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:52.675445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:52.782817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb40c62-84f0-4aa9-ad9c-58b69db476fb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:52.807442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e602f13-d04b-4d76-bb31-eb21526bb710 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:52.813017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:52.858665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:52.873734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:52.874338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:52.874651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:52.881798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:52.883311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:52.884067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:52.884818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:52.892890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:52.896148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:52.903873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 223/432] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:52 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:52.905977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:52.916571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:52.933659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:52.935526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 16:57:52.937296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:52.942035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 210/433] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:52.948446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:53.112087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd294212-4d20-4002-95d6-8bad8f85b4da None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:53.125076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:53.187759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87a901-b3b7-4fd0-af72-117f5913a06e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:53.207251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:53.209377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.210390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.211607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.213485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=5e8bf6cf3bb44be2a505004317684749) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:53.214365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:53.218982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:53.236276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 224/434] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:52 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=5e8bf6cf3bb44be2a505004317684749&role.id=55b0174dc86246af94379fcc1ef68040 => generated 611 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:53.244052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:53.245801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.246628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.247369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.249316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:53.250210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:53.253551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6561220-9d18-4c1b-ac49-30d56dceea9f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:53.275312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 211/435] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:53.286229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:53.474547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050a24e-ce0d-4291-be27-4c4b1b4f3539 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.475409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050a24e-ce0d-4291-be27-4c4b1b4f3539 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.476031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050a24e-ce0d-4291-be27-4c4b1b4f3539 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.481579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 212/436] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:53.491334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.491848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.492354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.494721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b78c1b8-ded9-4822-91ff-e5c90ef45a11 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:53.532637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:53.534442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:53.567423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba20d34-c6c6-4adb-8a23-081b3a72117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:53.568547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba20d34-c6c6-4adb-8a23-081b3a72117a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.569106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba20d34-c6c6-4adb-8a23-081b3a72117a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.569577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba20d34-c6c6-4adb-8a23-081b3a72117a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.580921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:53.645798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 213/437] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:57:53.656517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:53.680089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:53.709501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba20d34-c6c6-4adb-8a23-081b3a72117a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:53.713574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba20d34-c6c6-4adb-8a23-081b3a72117a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:53.852060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2c4ef-c403-486f-8d59-28210f361624 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:53.892466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 225/438] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:53.893953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97eb0b-f5e0-4971-9f93-661deb980371 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.894244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97eb0b-f5e0-4971-9f93-661deb980371 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.894516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97eb0b-f5e0-4971-9f93-661deb980371 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.895928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 226/439] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:53.901024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97eb0b-f5e0-4971-9f93-661deb980371 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:53.916045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed40ae4-303e-42ac-a8bb-e4e2493a9433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:53.918390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed40ae4-303e-42ac-a8bb-e4e2493a9433 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.918390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed40ae4-303e-42ac-a8bb-e4e2493a9433 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.918390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed40ae4-303e-42ac-a8bb-e4e2493a9433 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.939395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed40ae4-303e-42ac-a8bb-e4e2493a9433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3ed40ae4-303e-42ac-a8bb-e4e2493a9433 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:53.949547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:53.950925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:53.952303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 214/440] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:57:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:53.954868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:53.954868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:53.955355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:53.990168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:53.992353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:54.005728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:54.032973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:54.045608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97eb0b-f5e0-4971-9f93-661deb980371 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.093135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.094173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.094663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.095099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.097097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:54.097627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 215/441] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:54.098879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.099821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:54.109228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 227/442] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:53 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:54.112401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.233746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761776f-30df-47d0-a885-459b06e160b2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.252494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f84a-bd0a-4418-ba02-8bd8ea1a2057 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.282932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.283935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.284375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.284870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.286880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:54.289281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:54.298446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 228/443] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:54.298446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.299631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.300328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.300647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.300984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.302217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:54.302706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:54.305145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:54.309716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:54.325097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 216/444] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:54.325097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.347320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:54.429840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf65510-fb98-42d7-b569-4e95e387a06f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.472107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.472894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.473309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.473668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.486794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:57:54.493036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:54.494736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 229/445] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:54.496201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.502014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8d36e-ae67-447b-9ffe-6678319b6763 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.548759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.549424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.549704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.549978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.550798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=94d6523ed4be482abd3484d9adce8e0b, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:54.551107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:54.552941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:54.559405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 217/446] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=94d6523ed4be482abd3484d9adce8e0b => generated 611 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:54.560465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.598930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:54.680922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c0e0-9647-4ecd-a80d-ade1461a96fb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.709813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:54.742123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffacb18-0854-4219-bb63-3b7a4ed5886c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:54.780842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.781945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.782385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.782828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.804604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:54.806214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:54.807073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:54.807643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:54.809210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:54.809768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:54.813834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:54.831295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 218/447] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:54.840788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:54.913631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:54.915904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.019516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2ae8-a02a-443b-adb7-bd58b2bb08c1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:55.026283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:55.079952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:55.080414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:55.080830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:55.081247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:55.084128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 230/448] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:57:55.093208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:55.102167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:55.178125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:55.179927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.231021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b36c1c-41cb-42e8-958a-9de4fadc95cf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:55.286491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:55.287163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:55.288092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:55.288092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] PATH_INFO: `/v3/projects/20d464db83b0461b8a5f3a3fd94529c0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:55.289772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] RBAC: Authorizing `identity:get_project(project_id=20d464db83b0461b8a5f3a3fd94529c0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:55.291466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.294000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 231/449] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:55 2022] GET /identity/v3/projects/20d464db83b0461b8a5f3a3fd94529c0 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:55.301397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:55.346002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 219/450] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:57:55.354201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:55.361074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:55.425958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:55.457107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d33572-8295-4c73-af34-06593d74148f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:55.499942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:55.500983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:55.501507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:55.501962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] PATH_INFO: `/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:55.503895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] RBAC: Authorizing `identity:get_group(group_id=3d06338f42b049dc9a6d0d4b16c1ac79)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:55.506423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.509457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 232/451] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:55 2022] GET /identity/v3/groups/3d06338f42b049dc9a6d0d4b16c1ac79 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:55.515968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:55.516587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dde46e-76fa-4ce5-afce-25d792d27888 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:55.564125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:55.564870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:55.565297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:55.565687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] PATH_INFO: `/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:55.567127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] RBAC: Authorizing `identity:get_project(project_id=f28d3ad4783445e5a27ff264d5fb4f43)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:55.568738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.570777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 220/452] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:57:55 2022] GET /identity/v3/projects/f28d3ad4783445e5a27ff264d5fb4f43 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:55.579645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:55.644895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd93e-beae-4690-83dd-ca81b49da382 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:55.646813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:55.686548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:55.687193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:55.687479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:55.687753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:55.688584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] RBAC: Adding query filter params (group.id=3d06338f42b049dc9a6d0d4b16c1ac79, role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=20d464db83b0461b8a5f3a3fd94529c0) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:55.688896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:55.690480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.697885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 233/453] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:55 2022] GET /identity/v3/role_assignments?group.id=3d06338f42b049dc9a6d0d4b16c1ac79&scope.project.id=20d464db83b0461b8a5f3a3fd94529c0&role.id=55b0174dc86246af94379fcc1ef68040 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:55.752145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7539f-daf4-4c8c-ab8b-01cfcd7c4a07 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:55.770208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:55.812432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062bf55-47cd-410c-bc7f-f4373985b6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:55.813602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062bf55-47cd-410c-bc7f-f4373985b6be None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:55.814109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062bf55-47cd-410c-bc7f-f4373985b6be None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:55.814653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062bf55-47cd-410c-bc7f-f4373985b6be None admin] PATH_INFO: `/v3/groups/e48ee57702f642b984bd193983076f76` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:55.817038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062bf55-47cd-410c-bc7f-f4373985b6be None admin] RBAC: Authorizing `identity:get_group(group_id=e48ee57702f642b984bd193983076f76)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:55.820005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062bf55-47cd-410c-bc7f-f4373985b6be None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:55.823471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 221/454] 10.222.0.48 () {56 vars in 1180 bytes} [Tue Jul 26 16:57:55 2022] GET /identity/v3/groups/e48ee57702f642b984bd193983076f76 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:55.830321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:55.986100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9d344-9b69-41f2-9d9b-c4ba944d6dcb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:56.038609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:56.039359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:56.039812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:56.040202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:56.041162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] RBAC: Adding query filter params (group.id=e48ee57702f642b984bd193983076f76, role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=f28d3ad4783445e5a27ff264d5fb4f43) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:56.041530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:56.043176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa127-e398-4c13-a3c1-9a7f5bb5bea3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:56.052989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 234/455] 10.222.0.48 () {56 vars in 1376 bytes} [Tue Jul 26 16:57:55 2022] GET /identity/v3/role_assignments?group.id=e48ee57702f642b984bd193983076f76&scope.project.id=f28d3ad4783445e5a27ff264d5fb4f43&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:57:58.934876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b595f1-8b80-48ce-b002-1094c15f827b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:58.936309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b595f1-8b80-48ce-b002-1094c15f827b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:58.936456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b595f1-8b80-48ce-b002-1094c15f827b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:58.939543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 222/456] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:58.948523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:58.949095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:58.949405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:58.990906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:58.992948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:59.053984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:59.164816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 235/457] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:59.214147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b595f1-8b80-48ce-b002-1094c15f827b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:59.431482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b595f1-8b80-48ce-b002-1094c15f827b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:59.491487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:59.492068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:59.492351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:59.492645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:59.511977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:59.517711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-463fe197-37df-45f6-a56d-e57730d6270c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:57:59.535924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:57:59.537088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:57:59.538168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:57:59.538168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:57:59.538168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:57:59.538168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:57:59.538168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 223/458] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:57:59.545502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:59.735206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3fc4-ea44-42c1-988a-5a9bc37e301b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:59.735562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3fc4-ea44-42c1-988a-5a9bc37e301b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:59.735793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d3fc4-ea44-42c1-988a-5a9bc37e301b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:59.741856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 224/459] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:57:59.754565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b632-a644-4f6c-973d-63fb3352b5c8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:59.755088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b632-a644-4f6c-973d-63fb3352b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:59.755387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739b632-a644-4f6c-973d-63fb3352b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:59.810960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e739b632-a644-4f6c-973d-63fb3352b5c8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:59.813610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e739b632-a644-4f6c-973d-63fb3352b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:57:59.832738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543daa9-cd7d-4781-9f3e-a5998179cb51 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:57:59.881914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:59.910946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:57:59.911597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:59.911879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:59.912143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:59.912868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:57:59.913202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:57:59.914694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892da6b-12eb-4830-9c72-fe5145c92c71 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:57:59.940174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 236/460] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:57:59.945841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5903623-77af-43e6-92f1-3e4f0ee4d44f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:59.946428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5903623-77af-43e6-92f1-3e4f0ee4d44f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:59.946930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5903623-77af-43e6-92f1-3e4f0ee4d44f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:59.949097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 237/461] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:57:59.950845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5903623-77af-43e6-92f1-3e4f0ee4d44f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:57:59.970115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 225/462] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:57:59.971527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:57:59.971810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:57:59.972084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:57:59.984699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:57:59.996030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:57:59.997344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:00.099907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 226/463] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:00.101444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:00.144242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5903623-77af-43e6-92f1-3e4f0ee4d44f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:00.194748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:00.195771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.196264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.196749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.197770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:00.199749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:00.231652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 238/464] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:57:59 2022] POST /identity/v3/users => generated 248 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:00.234075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:00.281604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:00.299902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758cbb1-4679-4346-8d61-db4b0e88285c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:00.364316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:00.366269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.367061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.367790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.390800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:00.393849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:00.402185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-28f39178-6616-430b-89d4-e56fd3d63a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:00.446828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:00.447982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:00.450075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:00.450075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:00.450075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:00.450075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:00.450075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 227/465] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:00.451879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1160d-a1a1-4995-a119-17fef89b6e71 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.452299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1160d-a1a1-4995-a119-17fef89b6e71 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.452757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1160d-a1a1-4995-a119-17fef89b6e71 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.461779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 228/466] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:00.462301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a022-1316-47c9-91a3-bec443103fa1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:00.463458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1160d-a1a1-4995-a119-17fef89b6e71 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:00.526157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:00.527367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.527911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.528317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.544637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:00.547681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1b7a0ac4-369e-4c38-a376-e0d8119ec9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:00.571932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:00.572883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:00.573788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:00.573788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:00.573788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:00.573788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:00.573788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 239/467] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:00.575261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.575591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.576468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.603259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:00.604797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:00.642147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1160d-a1a1-4995-a119-17fef89b6e71 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:00.703497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 240/468] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:00.704683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:00.730209 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:00.731693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.732922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.733601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.735130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:00.735763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:00.747309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d768e8-663a-4278-905b-8ff51bb3ff59 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:00.782003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:00.785671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 229/469] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:00.793431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233924f3-7dc2-4611-8117-c30e2e00f04e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.793812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233924f3-7dc2-4611-8117-c30e2e00f04e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.793994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233924f3-7dc2-4611-8117-c30e2e00f04e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.796320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 230/470] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:58:00.798429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233924f3-7dc2-4611-8117-c30e2e00f04e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:00.805035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:00.844751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4997-948b-4959-bf67-ae12691e3fd6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:00.886084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:00.886969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:00.887397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:00.887842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:00.889092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:00.889707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:00.891290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:00.906068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 241/471] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:00.907293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:00.998770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233924f3-7dc2-4611-8117-c30e2e00f04e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:01.033279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d1258-44c9-4d0b-99b9-6da6a84a10e9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:01.061675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:01.068303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32651a-a00f-42af-aa67-44b23472e877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:01.069836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32651a-a00f-42af-aa67-44b23472e877 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:01.070283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32651a-a00f-42af-aa67-44b23472e877 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:01.070727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32651a-a00f-42af-aa67-44b23472e877 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:01.071791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32651a-a00f-42af-aa67-44b23472e877 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:01.074162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32651a-a00f-42af-aa67-44b23472e877 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:01.090933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:01.092185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:01.092503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:01.093197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:01.111069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:01.114731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:01.120339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 231/472] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:58:00 2022] POST /identity/v3/users => generated 248 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:01.127869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:01.128911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:01.129828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:01.151832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:01.153570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:01.155198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:01.155198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:01.155198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:01.155198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:01.161619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 242/473] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:01.163227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:01.174257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:01.176883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:01.322796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:01.361738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 232/474] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:01.362573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:01.419200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892a05c-bdbe-4451-9239-ef74ddaf294b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:01.510266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:01.511728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:01.512580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:01.514253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:01.515917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:01.519260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:01.560810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 243/475] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:58:01 2022] POST /identity/v3/users => generated 246 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:01.562669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:01.623483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:01.623823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0d487-aab3-4f85-9b46-5136c518de33 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:01.735958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:01.740439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:01.741507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:01.742211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:01.745907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:01.746924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:01.751432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:01.813390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 233/476] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:01.817589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:01.869413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22430f82-63a5-4dc2-bd22-561ddb9d385e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:01.939320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:01.944086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:01.945791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:01.946638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:01.947426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:01.949400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:01.952808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:01.967320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.sql.core [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 16:58:01.967320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 16:58:01.967320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [parameters: {'id': '33b0407beb8743f09d07da819f41a238', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 16:58:01.967320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result.read() Jul 26 16:58:02.002592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:58:02.004306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application yield resource Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise self.value Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application session.commit() Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 16:58:02.006128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.session.flush() Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application compat.raise_( Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 16:58:02.010665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:58:02.011656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result.read() Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:58:02.012671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [parameters: {'id': '33b0407beb8743f09d07da819f41a238', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:02.014542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 16:58:02.015379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:02.016207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 244/477] 10.222.0.48 () {60 vars in 1131 bytes} [Tue Jul 26 16:58:01 2022] POST /identity/v3/roles => generated 144 bytes in 448 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:02.017822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:02.069172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee113d6-f8c6-4ccf-af72-10944781aab3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:02.165218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:02.166353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:02.166786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:02.167208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:02.168118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:02.170548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:02.225316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 234/478] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:58:01 2022] POST /identity/v3/users => generated 249 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:02.244540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:02.261264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162e685-82ec-4016-bef8-4ffbaf5b644f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:02.336849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533b359-c45b-4f9b-9530-bcf2620c5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:02.337658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533b359-c45b-4f9b-9530-bcf2620c5907 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:02.337946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533b359-c45b-4f9b-9530-bcf2620c5907 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:02.338226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533b359-c45b-4f9b-9530-bcf2620c5907 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:02.352459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533b359-c45b-4f9b-9530-bcf2620c5907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e533b359-c45b-4f9b-9530-bcf2620c5907 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:02.359562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:02.361904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:02.363536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 245/479] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:58:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:02.374967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:02.586407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bee3f7-a4c0-4e5f-9687-9657b9879317 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:02.598717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:02.662634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:02.663737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:02.664234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:02.664728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:02.666124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:02.666659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:02.669877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abe2a3-f211-4c36-96c2-31890b02f7c1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:02.689201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 235/480] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:03.188113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1879a64-6447-4f35-9ada-d54a84593e5b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:03.188502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1879a64-6447-4f35-9ada-d54a84593e5b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:03.188634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1879a64-6447-4f35-9ada-d54a84593e5b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:03.190402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 246/481] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:03.196914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:03.197451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:03.197564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:03.220453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:03.221711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:03.353919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 236/482] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:03.398542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1879a64-6447-4f35-9ada-d54a84593e5b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:03.633810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1879a64-6447-4f35-9ada-d54a84593e5b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:03.714653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:03.715870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:03.716415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:03.717935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:03.745869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:03.759221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:03.760204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:03.761048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:03.762253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 247/483] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:03.773938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:04.043265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:04.052840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd3b8-84b0-42c7-9006-608af4370155 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:04.145895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:04.147006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:04.147519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:04.147988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:04.149454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:04.150019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:04.152762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605a23d-8688-4335-acbd-a330f6a816a9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:04.182930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 237/484] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:04.189597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:04.225486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:04.407494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955a1f0-2234-421d-9095-6285dec7649c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:04.416464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621856b6-3929-466e-9488-525e1fca702c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:04.417087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621856b6-3929-466e-9488-525e1fca702c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:04.417087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621856b6-3929-466e-9488-525e1fca702c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:04.418755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 238/485] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:04.424423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:04.424740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:04.425575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:04.461887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:04.463016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:04.463572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:04.464082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:04.470176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:04.471426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:04.478797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:04.480909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:04.501118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:04.502139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:04.503147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:04.503147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:04.503147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:04.503147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:04.506052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 248/486] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:04.513059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:04.633385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 239/487] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:58:04.651005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:04.663289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:04.709286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ceb78-e781-4fe7-adad-e378a2313ea0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:04.778092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:04.779364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:04.779652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:04.780097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:04.780937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:04.781380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:04.783312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:04.819059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:04.821634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 249/488] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:04.828369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:04.850632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc87e87-3ebb-4640-a72a-c9a19f0e1eba None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:04.933927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:04.935292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:04.935907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:04.936746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:04.964474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:04.976646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:04.978247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:04.979358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:04.986438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 240/489] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:04.992068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:05.107772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a887e8-9f49-494b-9023-2ce099bfcdc2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:05.165858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:05.186225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:05.186896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.187232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.187560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.202979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:05.204795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e9cad102-e2ec-44cd-86c2-8602a969ff3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:05.222900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:05.223914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:05.224856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 250/490] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:05.226316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f632e-c061-4c6b-bd6d-adfea1d03528 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.226599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f632e-c061-4c6b-bd6d-adfea1d03528 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.226909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f632e-c061-4c6b-bd6d-adfea1d03528 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.229008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 251/491] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:05.230405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f632e-c061-4c6b-bd6d-adfea1d03528 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:05.233396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b65f0-5efb-446b-b3bf-a992833e5831 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:05.292218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:05.293244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.293704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.294426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.295452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:05.295981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:05.298618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc05ca-236d-43fc-bdba-c0d2736ec648 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:05.313612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 241/492] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:05.315376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.315859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.316299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.348011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:05.350314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:05.377531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:05.386929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f632e-c061-4c6b-bd6d-adfea1d03528 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:05.437459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:05.438240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.438330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.438688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.439611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:05.439995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:05.441982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:05.484020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 242/493] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:05.484935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:05.505158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 252/494] 10.222.0.48 () {56 vars in 1137 bytes} [Tue Jul 26 16:58:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:05.516136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:05.574867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:05.633818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652d5c-107f-4a1b-9e82-e9533a304f8c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:05.649966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaa6d1-6850-40d3-9230-ecc820769471 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:05.673666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:05.674337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.674622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.674917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.687068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:05.688651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:05.690958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:05.708564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:05.710321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:05.712049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:05.712049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:05.712049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:05.712049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:05.712049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 243/495] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:05.712049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:05.725163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d66c6-6beb-441a-a80c-5fd8676a27a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:05.726494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d66c6-6beb-441a-a80c-5fd8676a27a6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.727195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d66c6-6beb-441a-a80c-5fd8676a27a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.728170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d66c6-6beb-441a-a80c-5fd8676a27a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.842159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad928fa7-de08-4c1b-a3de-8c0787e35443 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:05.866984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d66c6-6beb-441a-a80c-5fd8676a27a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:05.869313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d66c6-6beb-441a-a80c-5fd8676a27a6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:05.893665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab5c1d-37f6-42b5-bbcb-ef329d4c450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:05.894764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab5c1d-37f6-42b5-bbcb-ef329d4c450c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.895253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab5c1d-37f6-42b5-bbcb-ef329d4c450c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.895728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab5c1d-37f6-42b5-bbcb-ef329d4c450c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.915116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab5c1d-37f6-42b5-bbcb-ef329d4c450c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:05.920487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-deab5c1d-37f6-42b5-bbcb-ef329d4c450c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:05.920827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:05.921999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:05.923017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 244/496] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:05.924299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9c5d-7b1b-40e2-8ff4-c2de25f2547d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:05.924299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9c5d-7b1b-40e2-8ff4-c2de25f2547d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:05.924521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb9c5d-7b1b-40e2-8ff4-c2de25f2547d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:05.926074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 245/497] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:05.927216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9c5d-7b1b-40e2-8ff4-c2de25f2547d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:05.997339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 253/498] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 16:58:05.999414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:06.001574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864b66-193d-4047-acb4-b935938754c7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.002086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864b66-193d-4047-acb4-b935938754c7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.002641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864b66-193d-4047-acb4-b935938754c7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.007293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 254/499] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:06.007293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864b66-193d-4047-acb4-b935938754c7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.061471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb9c5d-7b1b-40e2-8ff4-c2de25f2547d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.101224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.102005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.102285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.102562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.103554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:06.103862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:06.105327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e097f212-bd28-4398-b02e-04892b7c6c64 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:06.119081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 246/500] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:06.120457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4db32d-b671-4c68-898d-1b3b7cb27f6f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.120753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4db32d-b671-4c68-898d-1b3b7cb27f6f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.121064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4db32d-b671-4c68-898d-1b3b7cb27f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.140530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6c4db32d-b671-4c68-898d-1b3b7cb27f6f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:06.141383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864b66-193d-4047-acb4-b935938754c7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.141814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c4db32d-b671-4c68-898d-1b3b7cb27f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:06.181062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.181762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.182071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.182386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.183070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:06.183379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:06.186113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:06.195503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 255/501] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:06.196384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.199728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:06.231632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 247/502] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:06.231632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.231632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.231632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.262130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:06.264375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:06.323838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8875da-d078-4ea5-963d-38916fb60fc0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.325660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:06.356231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 248/503] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:06.357889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.390103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.391149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.391665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.392275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.407023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:06.408799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:06.428545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:06.430411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:06.431364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:06.431364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:06.431364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:06.431364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:06.433322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 256/504] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:06.434695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.513718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ec016-d21c-47a3-a241-334033bd80a9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.523795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:06.573239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.574304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.574643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.576901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e868a7-b053-467d-a558-e9e021766e70 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.588181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:06.590702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:06.616288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:06.617523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:06.618443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 249/505] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:06.620057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.629383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.630182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.630544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.630937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.645401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:06.647249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:06.665182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:06.666310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:06.668066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:06.670372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:06.670372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:06.670372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:06.670372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:06.670372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 257/506] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:06.671135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.775284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12499f-f884-4c71-912e-e3bf7d4cb235 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.800677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00193556-9088-4804-b7de-ee448c705e09 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:06.814004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:06.829927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.830727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.831068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.831405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.847338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:06.858115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:06.859374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:06.867761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:06.868487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:06.868487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:06.868487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:06.868731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 250/507] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:06.868808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:06.868875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:06.880012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:06.883986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:06.912180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:06.914033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:06.915927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:06.915927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:06.915927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:06.915927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:06.915927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 258/508] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:06.915927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.015685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.025876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f212-16a1-4708-8b15-a7083310a940 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.067580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604ccef-6d2a-4107-ba6a-b4a6a3037be8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.069810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.070666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.071031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.071402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.072229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:07.072702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.074301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.089692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 251/509] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:07.091124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.116275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.117219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.117610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.117949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.118826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:07.119192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.121045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.155777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 259/510] 10.222.0.48 () {56 vars in 1137 bytes} [Tue Jul 26 16:58:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:07.155777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.176380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.240528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e9299-2dce-4e3c-841f-5296082aa460 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.290761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.292365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.292731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.293151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.293976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:07.294392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.297153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.310847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 252/511] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:07.311743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.312930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a821e74-5f66-4fd6-ad0b-0fbbe8ea3d31 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.331231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.376189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.377145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.377613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.378071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.378936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:07.379257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.380814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.391375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 260/512] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:07.392357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.465512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634ba9-a190-483a-bd39-a62fdd8807c6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.492865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.520624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1e53-c9a2-4bd3-b9c8-031276ee2e51 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.527272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.528338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.528873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.529409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.530740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:07.531281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.533987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.555099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 253/513] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:07.556255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.564361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.565220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.565552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.565861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.575107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.577607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:07.592741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:07.594068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 261/514] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:07.595688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.676949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.708761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6290322-a30d-4e00-9540-1d213939b736 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.750814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.751559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.751919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.752302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.754413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61736b-37ad-450f-8a58-74637c83ec67 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:07.807448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:07.808041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:07.808386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:07.808724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:07.819451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.821252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.824098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:07.839184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:07.841039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 262/515] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:07.843872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:07.853874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:07.855501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:07.960709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:07.990578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8b3e0-5ded-4e92-9a8f-1c6dcde6e531 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.004234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 254/516] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:08.005609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.038679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.039485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.039819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.040138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.054832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.056420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:08.074768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:08.077418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:08.078949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:08.078949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:08.078949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:08.078949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:08.078949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 263/517] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:08.078949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.139703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:08.152513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e14a7e-45d1-4a54-b4a5-f1fde499ba13 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.198001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.198786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.199151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.199548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.200294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.202478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.208660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc292a-cabe-4bd0-aee2-eed28553fd31 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.233543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 255/518] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:58:08 2022] POST /identity/v3/users => generated 247 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:08.234944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.260207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.261013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.261375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.261723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.262696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:08.263397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.265919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.301165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 264/519] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:08.301165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.312165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:08.473318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4c846-66b0-48fd-b772-58ea412ed0db None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.497659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f465f-461a-4639-b1ed-d7f603cdf0dd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.519895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:08.541399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.542416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.542871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.543301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.545324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:08.545825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.548250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.567378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.568280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.568695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.569181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.590562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.593902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.620273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 256/520] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:08.625875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:08.627487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:08.629185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:08.629185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:08.629185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:08.629185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:08.629185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 265/521] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:08.629185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.630853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.798482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:08.816211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d54f-0297-4db8-b46f-29e5b419d7f7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.824034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb59a7-51d8-4564-83a0-f3dc985d0725 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:08.868927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.870759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.871058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.871333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.873983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=0f33d17889ee4fe3a9692abe24bbdea9) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:08.874456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.875938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.889008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 266/522] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/role_assignments?user.id=0f33d17889ee4fe3a9692abe24bbdea9&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:08.889008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.921424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:08.922574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:08.923135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:08.923668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:08.925117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:08.925851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:08.928412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:08.953372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 257/523] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:08.954631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:08.957821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:09.149845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10750-1057-4f38-a79d-3dc8855aab83 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:09.150135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb2160-db60-4a55-82fd-915f22514ca2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:09.205000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:09.206395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:09.206985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:09.207679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:09.228813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:09.230604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:09.231393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:09.233374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:09.237661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:09.238317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:09.241156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:09.267755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 258/524] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:09.269637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:09.290632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:09.419302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:09.422380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:09.468662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9056907-7cb7-4a8c-b1a4-35046c8d95dd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:09.486325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:09.538273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:09.539999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:09.540770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:09.541523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:09.552606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:09.554835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:09.569064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:09.570052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:58:09.570961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:09.573889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 259/525] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:09.575115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:09.634521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 267/526] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:09.635458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:09.659055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:09.708915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139480c-72be-41a8-9c77-3a7489505854 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:09.756515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:09.757738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:09.758683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:09.758918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:09.776001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:09.778943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:09.801461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224291e2-a755-40a7-8457-bb7cf04f6afe None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:09.806803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:09.813041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 260/527] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:09.815734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:09.869206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:09.890261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:09.891395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:09.891935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:09.892466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:09.894018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:09.894617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:09.898156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:09.945119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 268/528] 10.222.0.48 () {56 vars in 1139 bytes} [Tue Jul 26 16:58:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:09.947486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:10.028139 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3e08e-747a-4d67-af67-8d399b737171 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:10.080673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:10.097113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:10.098162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:10.098759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:10.099284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:10.121274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:10.124126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:10.151722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:10.153475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:10.157416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:10.157416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:10.157416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:10.157416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:10.157416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 261/529] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:10.157416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:10.197872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c5ed0-a762-45ae-9c0a-cb2c7bb00458 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:10.265343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:10.266203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:10.266387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:10.266684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:10.269160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:10.269160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:10.269652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:10.284186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 269/530] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:10.285132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:10.345406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:10.379406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8750bc41-719c-41fd-bdc0-c1a0886cf384 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:10.415953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af185ce-b27a-466e-953c-8be3dc0b5f8e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:10.457178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:10.457819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:10.458105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:10.458379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:10.458971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:10.459177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:10.459479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:10.460827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:10.461233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:10.462285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:10.463155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:10.475665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 270/531] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:10.476605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:10.568879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:10.598176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a359a76-ab62-492a-a4f8-47318f1a4192 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:10.610522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:10.614071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:10.633368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:10.646131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:10.647167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:10.647308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:10.647459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:10.648284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=c2f2db6b70c241ec8bda9c96330f98c7) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:10.648642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:10.650555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:10.663754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 271/532] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:10 2022] GET /identity/v3/role_assignments?user.id=c2f2db6b70c241ec8bda9c96330f98c7&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:10.664265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:10.834054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81661d-12d5-4520-a4f9-6e413044145a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:10.874510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:10.875177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:10.875452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:10.875716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:10.889040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:10.889040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:10.890949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 262/533] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:10.899481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:10.922354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:10.923989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:10.925924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:10.929890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 272/534] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:10.936400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:11.041379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:11.129590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:11.196613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244de4c-6b75-4c37-85c8-50c728156598 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:11.221483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4443281-4111-467c-93f7-7a2a2a67f34e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:11.283308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:11.283984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.284259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.284534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.289034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:11.289672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.289951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.290213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.290976 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:11.291279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:11.292757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:11.311704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:11.313250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:11.326734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 273/535] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:11.333310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:11.338483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:11.347033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:11.353242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:11.353242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:11.353242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:11.353242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:11.353242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 263/536] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:11.353242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:11.553469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:11.553966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8383d98-ad57-4c5d-a21d-2211ec97224c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:11.564373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482d16f-2b33-4073-9d5a-815c6fe44f6a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:11.621531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:11.632084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:11.633459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.634094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.634679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.636321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:11.636695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:11.639492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c40b50-5a69-4b1c-92f5-fef39fc2fb8e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:11.645357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:11.646000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.646283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.646801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.647358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=a903f3771a62447d9c06fbeba414e9e3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:11.647669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:11.649186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e700c66-8468-4622-a997-ae891268c133 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:11.658045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 274/537] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:11 2022] GET /identity/v3/role_assignments?user.id=a903f3771a62447d9c06fbeba414e9e3&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:11.669013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3105fe1-de36-4866-8bbd-7e6b7682b92d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.669013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3105fe1-de36-4866-8bbd-7e6b7682b92d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.669013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3105fe1-de36-4866-8bbd-7e6b7682b92d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.669013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 275/538] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:11.669988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 264/539] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:11.670065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75f3eb-7859-4839-a96d-791aed15776b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.670065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75f3eb-7859-4839-a96d-791aed15776b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.670065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75f3eb-7859-4839-a96d-791aed15776b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.671327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 276/540] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:11.674149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75f3eb-7859-4839-a96d-791aed15776b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:11.675569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe048a-c313-414a-b15f-15d5f60f2bc8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.676089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe048a-c313-414a-b15f-15d5f60f2bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.676563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe048a-c313-414a-b15f-15d5f60f2bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.701724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-abbe048a-c313-414a-b15f-15d5f60f2bc8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:11.703026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-abbe048a-c313-414a-b15f-15d5f60f2bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:11.806093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 265/541] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:11.808291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.808580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.808860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.843421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:11.845179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:11.859445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75f3eb-7859-4839-a96d-791aed15776b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:11.929458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:11.931468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:11.932344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:11.932857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:11.941021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 266/542] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:11.942168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:11.952215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:11.956401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:11.977630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:11.979271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:11.980137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 277/543] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:11.986552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:12.006821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:12.061243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:12.141621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa126f-793e-42bf-bc03-713a6ad7b26b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:12.163413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1582c6-b8fb-47fd-b48e-da91a0f380cc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:12.184297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:12.184742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:12.185687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:12.186047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:12.186910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:12.187289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:12.188900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:12.209050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 278/544] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:12.210155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:12.240924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:12.242443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:12.243242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:12.244027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:12.268773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:12.282691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:12.284863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:12.286719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:12.290865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 267/545] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:12.292850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:12.319058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:12.378409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f7634-92a9-4b87-9316-75c26671f47e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:12.459313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:12.460358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:12.460824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:12.461354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:12.483539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:12.483736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85880fb6-31ab-4a36-92cf-ad70acb558e0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:12.493862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:12.494847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:12.495662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:12.496283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 279/546] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:12.497608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:12.507565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:12.553788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:12.554921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:12.555597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:12.555859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:12.557713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=ed1fb62d8f3f4a73ac9d11a37c62f99f) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:12.557983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:12.562006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:12.577194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 268/547] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:12 2022] GET /identity/v3/role_assignments?user.id=ed1fb62d8f3f4a73ac9d11a37c62f99f&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:12.577194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:12.700411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3942bd-d1d1-4ab2-a2c5-a6a2e44a8a1e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:12.833642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:12.850188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:12.851424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:12.851923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:12.852459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:12.854187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:12.854726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:12.861465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632e436-cd39-4fb8-be45-4a07a1f77027 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:12.865682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:12.895509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 280/548] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:12.901734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:12.944607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:12.947358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:12.947850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:12.948323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:12.954872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:12.957429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:12.963494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:12.995861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 269/549] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:13.004753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:13.167169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:13.293223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee0a54-0e07-4350-8ae0-f49c1e9ea3f7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:13.373916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:13.375056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:13.375545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:13.376023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:13.389434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153445-b54c-4b10-a0b5-6d4d474ad9ff None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:13.426175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:13.428851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:13.461400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:13.462361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:13.463311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:13.463311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:13.463311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:13.463311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:13.463963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 281/550] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 563 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:13.475721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:13.510550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:13.511614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:13.512106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:13.512608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:13.538178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:13.540116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:13.578166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:13.579220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:13.580140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:13.580140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:13.580140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:13.580140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:13.582413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 270/551] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 579 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:13.588570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:13.595266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:13.726566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02fac5-aae1-49f4-9881-c0458d073ecb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:13.779146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:13.794965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aca30d-a1af-473f-a35f-ac669f8e4c87 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:13.870459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:13.871543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:13.872042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:13.872593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:13.875462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:13.876007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:13.878659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db83115-c07f-4382-88f1-50e47a7b0e39 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:13.913301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:13.914817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:13.914817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:13.915117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:13.916286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:13.916746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:13.919215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2cf00-b621-420d-b846-e06215e7caed None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:13.920891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 282/552] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:13.923187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100368-8fa1-4f25-b0c6-d3a25b2bb823 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:13.923924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100368-8fa1-4f25-b0c6-d3a25b2bb823 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:13.924373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0100368-8fa1-4f25-b0c6-d3a25b2bb823 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:13.933091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 283/553] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:13.934222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100368-8fa1-4f25-b0c6-d3a25b2bb823 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:13.947009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 271/554] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:13.957626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:13.958426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:13.958908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:13.991460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:13.993380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:14.135448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 272/555] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:14.137234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.166183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0100368-8fa1-4f25-b0c6-d3a25b2bb823 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:14.185761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:14.229069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:14.230104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.230594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.230877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.240242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:14.241870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-fcf518c2-f1e5-43be-903f-3c6434760947 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:14.255638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:14.257356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:14.259030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:14.261684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 284/556] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:14.264095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77150860-3cc7-4028-baeb-63c7229ee0e8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.264449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77150860-3cc7-4028-baeb-63c7229ee0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.264731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77150860-3cc7-4028-baeb-63c7229ee0e8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.266212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 285/557] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:14.271243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77150860-3cc7-4028-baeb-63c7229ee0e8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.291292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:14.318924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524fa59-e7ce-4471-a3de-9fd51dd71b41 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:14.372561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:14.374026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.374316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.374595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.383685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:14.385335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:14.399085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:14.400770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:58:14.404629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:14.406712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 273/558] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:14.408070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.425590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77150860-3cc7-4028-baeb-63c7229ee0e8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:14.476169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0688f-4a9d-48be-9dc8-a0ac4dcd077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:14.476835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0688f-4a9d-48be-9dc8-a0ac4dcd077a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.477164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0688f-4a9d-48be-9dc8-a0ac4dcd077a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.477424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0688f-4a9d-48be-9dc8-a0ac4dcd077a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.492011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0688f-4a9d-48be-9dc8-a0ac4dcd077a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b2e0688f-4a9d-48be-9dc8-a0ac4dcd077a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:14.497176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:14.499318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 286/559] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:14.501154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.502645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.502645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.520418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:14.521781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:14.548478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:14.549267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509432f-d769-4ca7-9ec3-911159ea4f38 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:14.607658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:14.608357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.608643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.608921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.610195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:14.610195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:14.611553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:14.626415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 287/560] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:14.626415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.636219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:14.651996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 274/561] 10.222.0.48 () {56 vars in 1135 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:14.653020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.786340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339480b-854e-4981-8c69-e71b85c1eddc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:14.826656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b1e4-8c3b-410b-b407-0e1d4c63ae7c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:14.860781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:14.861925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.862433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.862926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.864311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:14.864873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:14.867589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:14.877100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:14.877953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:14.878432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:14.878880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:14.879898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:14.880248 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:14.882025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:14.885840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 275/562] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:14.887746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.911056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 288/563] 10.222.0.48 () {56 vars in 1137 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:14.911951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:14.949610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:15.011183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:15.091115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a94e7-e954-4bae-8178-fc748e694df3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.140231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.140919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.141263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.141547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.144150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243aede5-3a31-488d-807f-089f87e2db6b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.155810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:15.161656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:15.171769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 276/564] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:15.173443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:15.213636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.214766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.215263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.215761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.334849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:15.337359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:15.355490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695dde2-b8a0-4182-8d60-7fe105e06a91 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.364927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:15.400295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:15.429151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.429829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.430110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.430390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.454668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:15.459109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:15.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:15.491347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:15.492985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:15.492985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:15.492985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:15.492985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:15.493459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 289/565] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:15.493459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:15.499579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 277/566] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:15.501107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:15.619749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c32ae-d2df-4326-89cc-e97735c73f76 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.636112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:15.661073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.661818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.662100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.662373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.674223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:15.675832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f1f91149-a8df-4df9-8ba6-e07815212be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:15.694288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:15.695747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:15.696801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:15.696801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:15.696801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:15.696801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:15.696801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 290/567] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:15.696801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a13c42-663b-48b5-ba4f-3bd764c77afc None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.697316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a13c42-663b-48b5-ba4f-3bd764c77afc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.697316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a13c42-663b-48b5-ba4f-3bd764c77afc None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.698616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 291/568] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:15.699346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a13c42-663b-48b5-ba4f-3bd764c77afc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:15.710714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c4498-1ec2-4770-90b7-1667f0a54791 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.759121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.759766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.760049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.760330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.761150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:15.761437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:15.762861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:15.772988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 278/569] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:15.773880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:15.827962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:15.861116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a13c42-663b-48b5-ba4f-3bd764c77afc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.913566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293985e0-6cc3-4e4c-adc5-e9a8f2719980 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:15.923120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.924243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.924745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.925278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.945989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:15.948748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:15.951834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:15.952682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.952990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.953279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.954026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:15.954325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:15.955753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ff694-2251-49af-ae29-adb79429fbd3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:15.968388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:15.977688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:15.979723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:15.979723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:15.979723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:15.979723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:15.979961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 279/570] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:15.980107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 292/571] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:15.980107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:15.980340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:15.980340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:15.980340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:15.994334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:15.995866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:16.004061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.096076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 280/572] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:16.097770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.104340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1808e6-dc84-44e2-988e-7508cacc6ce9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.140002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.159900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.160906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.161408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.161867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.163084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:16.163648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.165794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.181948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 293/573] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:16.182923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.252544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdb077-a581-438c-b9d6-7459bafdce04 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.296460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.301168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.302051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.302443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.302847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.316431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.318322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.326507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0845bb13-fdc9-48e9-a12e-6acec76c0573 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:16.337767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:16.339514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:16.341493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:16.341493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:16.341493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:16.341493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:16.341493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 281/574] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:16.341493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.376553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.377312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.377633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.378026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.388264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.390425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:16.410640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:16.412563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 294/575] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:16.414958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.459368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.493222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920c545-4ade-4995-bfe4-4a2d513d13c5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.531536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.532173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.532462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.532743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.533497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:16.533810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.535307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.549786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 282/576] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:16.550690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.568780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d3049-e7b5-4b9f-b469-8cf98883b6c0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.614101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.614766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.615047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.615317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.624550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.630748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:16.635993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:16.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:16.637932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 295/577] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:16.639047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.708779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b11dd-d3cc-4263-b2e6-fa19f2f52119 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.755382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.756103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.756383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.756649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.760768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.769587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.771342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.785710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d669343-bdd1-4e12-8b41-6773818c3b60 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:16.786881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:16.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 283/578] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:16.790329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.841359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.841989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.842267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.842712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.843427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:16.843726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.845297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.860883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 296/579] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:16.861788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:16.934320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:16.943586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e45413-5223-4d65-b93e-2661a8048258 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:16.983101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:16.983722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:16.983999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:16.984277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:16.985160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:16.985469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:16.987104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:16.993819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd949d-9e8c-45c6-874a-0cc0e77bff32 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.009529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 284/580] 10.222.0.48 () {56 vars in 1137 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:17.010941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.053315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.054468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.054887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.055328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.065213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.067392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:17.082307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:17.084220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 297/581] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:17.086611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.098601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:17.161707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376959af-39fc-4f5b-93bc-fd836a4d798e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.205851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.206574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.206963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.207295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.208317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:17.208683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.210287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.219288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04432b-e856-46db-9e30-2459951f23da None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.220896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 285/582] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:17.222208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.245942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:17.271841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.272561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.272901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.273265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.274111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:17.274479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.276154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.292556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 298/583] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:17.293875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.352882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59227bc5-bebf-44f0-a6a0-1a34c5339dc7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.379759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:17.403600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.404433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.404909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.405307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.418452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.421130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:17.441729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:17.442981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:17.444783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 286/584] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:17.446693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.456784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de30b0-bc0f-48a5-8968-cf2cb106ec1a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.501094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.501932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.502308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.502639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.517971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.519681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:17.538362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:17.540311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:17.542100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:17.542100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:17.542100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:17.542100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:17.542100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 299/585] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:17.542100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.579503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a6e14-5ba1-47a6-802f-f0888399f71c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.583904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:17.629862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.630918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.631381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.631880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.633366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:17.633939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.636254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.669274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cb51d-a014-4ebe-80df-a21178144900 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.672040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 287/586] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:17.673746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.709540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.710204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.710505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.710757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.722513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.724005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.735954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:17.750115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:17.751844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:17.754138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:17.754138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:17.754138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:17.754138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:17.754138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 300/587] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:17.754138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.853839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17511c8d-ef83-479a-87d1-58e3eef9a5ba None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.861829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:17.893128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a83207-e44a-4c6f-a642-49b3ebdd0f81 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:17.897304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.898258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.898572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.899028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/0f33d17889ee4fe3a9692abe24bbdea9/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.922016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=0f33d17889ee4fe3a9692abe24bbdea9, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.923822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.942466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 288/588] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:17 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/0f33d17889ee4fe3a9692abe24bbdea9/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:58:17.944134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:17.950426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:17.952299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:17.952760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:17.953252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:17.955173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:17.955422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:17.957425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:17.998668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 301/589] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:18.001611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:18.134869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:18.167125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4764091-1a61-45fd-93ec-6821a2e6dfff None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.190446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a6b14-c3d9-40d4-be71-b69301f04dcd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.226740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.227572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:18.227713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:18.227981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:18.228940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:18.229294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:18.230801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:18.244601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 289/590] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:18.245527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:18.262393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.263039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:18.263904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:18.264318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:18.265510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=0bfb7deb3aaf48a6992edb1b01b17aff) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:18.265831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:18.267340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:18.285200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 302/591] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:17 2022] GET /identity/v3/role_assignments?user.id=0bfb7deb3aaf48a6992edb1b01b17aff&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:18.285200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:18.303068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:18.408203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2200d-8bb9-444b-add6-8d689534e72b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.481308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.483252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:18.483886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:18.484377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:18.484597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e182ef-5b21-4b58-a1a0-786980f0c2c9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.485920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:18.486468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:18.489288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:18.515552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 290/592] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:18.516502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:18.528992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:18.544619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.545978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:18.546493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:18.547016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:18.569721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:18.572437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:18.599857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:18.600849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:18.601813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:18.601813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:18.601813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:18.601813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:18.607664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 303/593] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:18.609462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:18.656168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221570-138b-48bc-bccf-32ec02269304 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.697727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.698776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:18.699271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:18.699763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:18.709058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:18.710942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:18.724191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:18.725208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:18.726148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:18.728670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 291/594] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:18.729809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:18.742562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:18.817547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:18.860401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e98b6-c518-4633-95ab-2b391c9cf99b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.901489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3022a52-cd9b-4894-806c-8f1eafb89675 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:18.948128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.949571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:18.950071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:18.950603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:18.968860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:18.972354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:18.994022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:18.995077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:58:18.996044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:18.998569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:18.999630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:19.000146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:19.000652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:19.001682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:19.002226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:19.003874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:19.014577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 304/595] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:19.015884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:19.030665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 292/596] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:19.032951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:19.266889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e42b8-3f84-4f42-bb06-ffce6801b9a4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:19.272244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fa532-024a-4e12-974f-958bdffad948 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:19.304160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:19.315173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:19.332006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:19.335408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:19.336161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:19.336710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:19.340024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:19.340649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:19.343577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:19.390587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 293/597] 10.222.0.48 () {56 vars in 1139 bytes} [Tue Jul 26 16:58:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:19.391863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:19.404848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:19.409841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:19.411279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:19.414196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:19.415898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:19.418474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:19.425747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:19.469446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 305/598] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:19.470872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:19.615712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57117e3-d99e-4ed4-bf19-028b830e1d56 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:19.658676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:19.659862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:19.660363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:19.660844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:19.669696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:19.671673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:19.686691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:19.687667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:19.688575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:19.696522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 294/599] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:19.698008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:19.736333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1952-70c0-4462-9e56-09ffcf536ef6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:19.761362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:19.807622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:19.808739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:19.809435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:19.809963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:19.832224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:19.835073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:19.835222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7af2b-538a-492c-8496-118d7ebd0db3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:19.867240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:19.868183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:19.869129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:19.869129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:19.869129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:19.869129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:19.870089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 306/600] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:19.871756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:19.888419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:19.889805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:19.891083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:19.891599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/c2f2db6b70c241ec8bda9c96330f98c7/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:19.914764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=c2f2db6b70c241ec8bda9c96330f98c7, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:19.917503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:19.932209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:19.943681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 295/601] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:19 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/c2f2db6b70c241ec8bda9c96330f98c7/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:58:19.943681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:20.144547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:20.154395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f730f0c-f309-47a9-b3bd-6c44be63c359 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:20.187012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a0bc6-a52a-423c-a897-d8caf2093fe2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:20.236638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:20.237809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.238312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.238843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.239842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:20.240376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:20.242192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:20.263210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:20.264209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.265471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.265471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.266286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:20.266765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:20.273203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:20.274266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 296/602] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:20.281869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:20.295426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 307/603] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:20.301681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:20.429384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:20.438302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2260a-b1be-40f1-995f-211d779f529a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:20.509715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:20.510418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.510689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.510958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/a903f3771a62447d9c06fbeba414e9e3/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.540432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bcb84-e765-411f-a1f7-e459b951f613 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:20.547297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=a903f3771a62447d9c06fbeba414e9e3, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:20.548810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:20.584590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 308/604] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:20 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/a903f3771a62447d9c06fbeba414e9e3/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:58:20.597098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:20.597830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.598118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.598384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.613365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:20.615153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-bd5e8321-9b99-4443-b18e-55199686eb86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:20.628875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:20.630443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:20.631874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:20.639405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 297/605] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:20.645591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:20.815494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06698ca-bae1-462b-9ddf-54d334c9a754 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.815797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06698ca-bae1-462b-9ddf-54d334c9a754 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.815939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06698ca-bae1-462b-9ddf-54d334c9a754 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.817692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 298/606] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:20.818840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b6813-3b6e-4022-b894-cf707644555e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:20.823790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.824101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.824412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.855031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:20.857950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:20.880884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:20.926085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:20.927907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:20.928890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:20.929676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:20.931357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:20.932166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:20.935937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:20.951818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:20.965725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 309/607] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:20.976274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:20.998899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 299/608] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:21.021183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:21.180605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1dcc7-fcfa-4553-bd13-3ced2481db38 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:21.224538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:21.225556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:21.225985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:21.226440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:21.239539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:21.245452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:21.246745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:21.247832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:21.250763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 300/609] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:21.258199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:21.269357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:21.272698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac7dcd-7782-4c8a-9092-4ac8c1c72625 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:21.367260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11301987-8b6d-4166-8da8-f262bec3794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:21.368446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11301987-8b6d-4166-8da8-f262bec3794a None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:21.368999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11301987-8b6d-4166-8da8-f262bec3794a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:21.369507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11301987-8b6d-4166-8da8-f262bec3794a None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/ed1fb62d8f3f4a73ac9d11a37c62f99f/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:21.411163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11301987-8b6d-4166-8da8-f262bec3794a None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=ed1fb62d8f3f4a73ac9d11a37c62f99f, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:21.415643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11301987-8b6d-4166-8da8-f262bec3794a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:21.455177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 310/610] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:20 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/ed1fb62d8f3f4a73ac9d11a37c62f99f/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:58:21.455637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e5f6e-a274-4e46-af0f-0dbf5a26aefd None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:21.455878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e5f6e-a274-4e46-af0f-0dbf5a26aefd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:21.456241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e5f6e-a274-4e46-af0f-0dbf5a26aefd None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:21.460865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 311/611] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:21.466126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fd878-9605-4116-930a-b11e3f32c4be None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:21.471508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:21.472004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:21.472487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:21.537715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:21.540075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:21.587784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:21.588925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:21.589437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:21.589869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:21.591162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:21.595385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:21.598765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:21.633582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 301/612] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:21.642360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:21.711657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:21.787663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 312/613] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:58:21.821085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:21.835549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:21.915245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade551b-f2ba-45f9-9e0e-cee9b9a3673b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:22.005608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:22.006639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:22.007117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:22.007604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:22.050949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:22.059459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:22.096483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:22.097606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:22.098562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:22.098562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:22.098562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:22.098562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:22.099298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 302/614] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:22.109666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:22.164150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881362bf-c409-48d0-9402-7c1869d8bf33 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:22.258461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:22.259494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:22.259974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:22.260453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:22.288643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:22.301558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:22.303850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:22.305969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 313/615] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:22.315906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:22.409523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:22.440658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e76035-cf54-44b4-8f1c-024aa94c6a73 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:22.477113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:22.522559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614fb8d-a28b-4d36-a09b-9026cbf89702 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:22.545355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:22.546411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:22.546897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:22.547379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:22.548651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:22.560867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:22.563427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:22.573078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:22.581881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:22.582312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:22.582697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:22.583573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:22.583992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:22.585622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:22.586631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 303/616] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:22.594567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:22.603171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 314/617] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:22.613866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:22.829838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:22.838110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec48ca-ed9c-4331-9640-ab8a1dd42ddd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:22.929488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:22.930591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:22.931073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:22.931520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:22.974126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:22.976899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:22.985161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8d1cbd-2522-469e-8d0a-f65f10bf91ac None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:23.007558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:23.008547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:23.009670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:23.009670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:23.009670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:23.009670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:23.012050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 315/618] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:23.020441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:23.108078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:23.109021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.109485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.109941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.128004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:23.130311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ee5e242a-2103-470b-8ea5-ac938e7bedfb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:23.146094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:23.147143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:23.148153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:23.149392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 304/619] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 554 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:23.149863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1176a-e23b-4214-9282-2e7e08b20282 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.150149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1176a-e23b-4214-9282-2e7e08b20282 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.150426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1176a-e23b-4214-9282-2e7e08b20282 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.151976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 305/620] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:23.157880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.158153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.158412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.183789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:23.186488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:23.232757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:23.267418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608315-bb34-4da3-99c1-df73c2c62909 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:23.301117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 306/621] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:58:23.302476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:23.328264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:23.328945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.329223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.329490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.332036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:23.332587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:23.334438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:23.350888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 316/622] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:23.351600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:23.374445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:23.449510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41144f59-7769-400d-8d07-e9588e3228bb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:23.483121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db8650-a50c-410f-8e68-92edfe3a3cc0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:23.490889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:23.491695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.492076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.492451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.493422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:23.493827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:23.496132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:23.531728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 307/623] 10.222.0.48 () {56 vars in 1135 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:23.533777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:23.534802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:23.535593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.536002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.536374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.562637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:23.572778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:23.574906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:23.576596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 317/624] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:23.578845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:23.604701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:23.732570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:23.757651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5375389-b802-4ae2-87c9-a9e8560893f7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:23.799957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7b178-3209-4375-b695-4a3162742600 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:23.824803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:23.825935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.826406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.826868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.840526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:23.844121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:23.861359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:23.862500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:23.862983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:23.863464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:23.867604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:23.868644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 16:58:23.870017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:23.873459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 308/625] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:23.874985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:23.881768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:23.883815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:23.909579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:23.911457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:23.913432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:23.913432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:23.913432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:23.913432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:23.913432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 318/626] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:23.913432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.046155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:24.075179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9b4f1-2f82-4837-8de0-5624e25a8886 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.082332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c625365-3fa3-485c-a8f5-c4314cab363f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.122342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.123641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.123923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.124298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.125085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:24.125386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.126922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781860b-ab79-4de3-8584-80f1950bf850 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.131686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:24.142106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 309/627] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:24.142106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a684-a93c-4f22-8883-9c96e63db515 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.142106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a684-a93c-4f22-8883-9c96e63db515 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.142106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a684-a93c-4f22-8883-9c96e63db515 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.142106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 310/628] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:24.142106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a684-a93c-4f22-8883-9c96e63db515 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.161845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.162833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.163260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.163677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.165211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:24.165637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.168303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.208797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 319/629] 10.222.0.48 () {56 vars in 1137 bytes} [Tue Jul 26 16:58:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:24.208797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.289063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a684-a93c-4f22-8883-9c96e63db515 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.348840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.349898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.350355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.350808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.352712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:24.353566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.356052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1270719-237c-48d7-b56c-c5545c09895e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.374052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441887d-b483-4234-8cca-a1eaffacfedb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.378902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 311/630] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:24.380782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.381346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.381827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.410627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:24.412602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:24.416526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:24.444661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.445390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.445672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.445963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.460277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.461878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:24.483596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:24.484687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:24.485661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:24.485661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:24.485661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:24.485661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:24.490710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 320/631] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:24.490710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.522468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 312/632] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:24.523384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.540216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:24.625189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f578b6-cd81-495b-9316-9076b7b15a35 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.679854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1ac08-f841-408d-b1a7-d72b24791751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.680660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1ac08-f841-408d-b1a7-d72b24791751 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.681093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1ac08-f841-408d-b1a7-d72b24791751 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.681503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1ac08-f841-408d-b1a7-d72b24791751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.684482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc8f82-f935-472d-835a-3f04b94d1091 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.731387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.732401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.732775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.733200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.742377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.744289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.746711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:24.765996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:24.767013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:24.767909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:24.768353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 313/633] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:24.769443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.783976 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1ac08-f841-408d-b1a7-d72b24791751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.785895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1ac08-f841-408d-b1a7-d72b24791751 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.834180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:24.894814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff151-a230-4111-ac0d-b3a8cfeab30d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:24.911746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 321/634] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:24.913769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da8dd5-8a38-4635-91f0-8d5a0d5f9824 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.914204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da8dd5-8a38-4635-91f0-8d5a0d5f9824 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.914630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46da8dd5-8a38-4635-91f0-8d5a0d5f9824 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.916642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 322/635] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:24.917650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da8dd5-8a38-4635-91f0-8d5a0d5f9824 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:24.940446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:24.941128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:24.941400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:24.941665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:24.942378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:24.942679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:24.944080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:24.957881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 314/636] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:24.958775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.037362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.080515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46da8dd5-8a38-4635-91f0-8d5a0d5f9824 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.102718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245b43-88df-4650-a6bf-d6f45f7bb08d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.131821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.132480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.132757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.133063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.135309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.147050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.147569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.147782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:25.147922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.148119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.148889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:25.149250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.150742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c06bb5-76b8-442d-a954-6a4757f8f671 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:25.152856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:25.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 323/637] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:25.155420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.167901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 315/638] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:25.170136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.170492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.170679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.191618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:25.192849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:25.274744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 316/639] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:25.275696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.284363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bb090-995b-4a5c-9ab1-913599a40e9e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.328394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.329207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.329674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.330022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.341495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.342981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.347769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:25.359576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:25.361356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:25.363795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:25.363795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:25.363795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:25.363795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:25.363795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 324/640] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:25.363795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.416263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ae83c-591d-4db0-830d-ff43a6d3c284 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.425843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.458443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.459099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.459375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.459647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.468827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.470364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:25.483992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:25.486241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:58:25.488303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:25.489797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 317/641] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:25.491023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.493091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc6e95-0b60-49d9-bf8d-540bc2906d2a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.532914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.533589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.533868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.534139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.534871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:25.535167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.536601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.546849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 325/642] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:25.547730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.626022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773813f7-530e-4aa3-9e3d-1729ccb9bce7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.637089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.666074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e58cb6-78c6-4b92-9df9-b8f8422e72eb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.673448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.674254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.674621 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.674983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/0bfb7deb3aaf48a6992edb1b01b17aff/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.680036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.692679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=0bfb7deb3aaf48a6992edb1b01b17aff, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.694401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.709442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 318/643] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:25 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/0bfb7deb3aaf48a6992edb1b01b17aff/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:58:25.710714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.711973 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.712781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.713168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.713584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.727428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:25.734473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:25.736421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 326/644] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:25.745899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.870353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0787e-fb2e-45c5-9b14-d9b58e881ad0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.884553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88a6ce-892b-49ce-b2d0-68786e58e078 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:25.911404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.912093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.912383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.912658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.913548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:25.913861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.915423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.931559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 319/645] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:25.932555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:25.947536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:25.948241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:25.948544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:25.948819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:25.949718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:25.950029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:25.952114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:25.963869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:25.973413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 327/646] 10.222.0.48 () {56 vars in 1137 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:25.974406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.012798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:26.084615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69f0df-46c8-4afa-8019-947ccd4aba22 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.126858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b45f49-d49a-42f4-a9dd-f71d5f89023c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.137638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.139662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.140332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.141021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.160526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.163309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:26.188917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:26.189919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:26.190842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:26.190842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:26.190842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:26.190842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:26.190842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 320/647] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:26.192174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.201211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.202302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.202786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.203349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.204400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:26.204987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.207184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:26.227809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 328/648] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:26.227809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.292824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:26.332175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039f7b0-ccc0-4e57-a9c3-e03961b76ef4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.379305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.380184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.380623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.381117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.383370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:26.395372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.400734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:26.405808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737369c-a984-4ea5-b541-d32755e25b63 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:26.421968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:26.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:26.424371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:26.433380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 321/649] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:26.434734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.475980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.477099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.477565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.478088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.609266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.611997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:26.614146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7147d-11b5-45ab-80ed-d75fa29435a3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.664302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:26.673264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.673910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.674195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.674468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.675181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:26.675484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.676898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:26.692163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 322/650] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:26.693083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.783797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:26.810816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 329/651] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:26.810816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.833706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7e5e6-7ed4-4cd3-9510-76e250ff5c79 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.897515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.898960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.899552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.900151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.921351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.924202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:26.937209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be884d-282a-4d50-b011-2cf55ffa2dc1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:26.955050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:26.956231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:26.957175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:26.957175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:26.957175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:26.957175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:26.957404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 323/652] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:26.958951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:26.986904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:26.987550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:26.987836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:26.988117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:26.988896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:26.989243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:26.990764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.004923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 330/653] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:27.005661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.037648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:27.078153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:27.110718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745405bb-c67b-4c1d-af4a-f25277715bf2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.148309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.148980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.149274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.149547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.158174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.159651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:27.173428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:27.175411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 324/654] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:58:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:27.177387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.178678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f077fda-6de6-4e60-b831-2b72ab549ba8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.227585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.228886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.229395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.229855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.250628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.252683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:27.275977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:27.277317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:27.278414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:27.278414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:27.278414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:27.278414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:27.284410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 331/655] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:27.284410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.338047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb43467-f404-47aa-830a-780ad0c4dec8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.341103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:27.390403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.391522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.392025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.392541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.393770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:27.394169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.396603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.417789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 325/656] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:27.417789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.442062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:27.445493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7442c5-da03-490d-bc32-86a373ed534a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.493399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.494082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.494440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.494723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.495728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=0f33d17889ee4fe3a9692abe24bbdea9) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:27.496041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.497688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.514573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 332/657] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/role_assignments?user.id=0f33d17889ee4fe3a9692abe24bbdea9&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:27.516214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.565027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cf0ab-bc42-4da5-9f14-6558cf361000 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.635537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.636678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.637297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.637850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.639628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:27.640201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.641949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.676572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 326/658] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:27.682799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.698488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b1036-fae3-400d-a2eb-d6ad2afbb7cd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.725504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:27.756257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.756257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.756257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.756257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.756691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:27.756691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.756691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.777120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 333/659] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:27.779415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.834512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:27.858958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a31ed-8485-4af4-834b-0e53ab031ca6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:27.935185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:27.936030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:27.936449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:27.936777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:27.945847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:27.947358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:27.963971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:27.966298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 327/660] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:27.969865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:27.997832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435334b5-42c9-4e13-95c0-27f5f2dc0b80 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:28.067615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:28.069073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.069769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.070308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.121088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:28.158759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fbb2b-4ae6-4ab4-a61d-bb4da774910b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:28.178463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:28.181943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:28.214794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:28.215371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.215653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.215932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.224871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:28.226402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:28.237130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2e3c4d1e-2173-4d11-959b-a534846613ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:28.240454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:28.242837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:28.244740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:28.246827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 328/661] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:28.248738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63369-be71-4937-bab6-a77c01a7fb0e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.249053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63369-be71-4937-bab6-a77c01a7fb0e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.249337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb63369-be71-4937-bab6-a77c01a7fb0e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.250962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 329/662] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:28.251596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63369-be71-4937-bab6-a77c01a7fb0e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:28.398258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb63369-be71-4937-bab6-a77c01a7fb0e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:28.426218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 334/663] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:28.428067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:28.455368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:28.456024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.456464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.456622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.457610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:28.458050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:28.459571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5846f-9bc8-4836-b3ae-0ebfd69e4081 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:28.473214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:28.486939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 330/664] 10.222.0.48 () {56 vars in 1139 bytes} [Tue Jul 26 16:58:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:28.488571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48a493-2875-457e-9532-baab789e1614 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.488998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48a493-2875-457e-9532-baab789e1614 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.489378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48a493-2875-457e-9532-baab789e1614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.511418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cc48a493-2875-457e-9532-baab789e1614 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:28.512845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cc48a493-2875-457e-9532-baab789e1614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:28.626792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 331/665] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:28.628477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48a493-2875-457e-9532-baab789e1614 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:28.629726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ca948-9271-4604-82df-ddae8576daf0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:28.686210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:28.710521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:28.711561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.712398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.712398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.713516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:28.714017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:28.717467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:28.747946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 335/666] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:28.747946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:28.828247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48a493-2875-457e-9532-baab789e1614 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:28.872050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:28.885327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:28.886587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.887101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.887592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:28.901880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc04c3b-116f-4adf-ac31-7aef81055a40 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:28.907069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:28.908601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:28.928260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:28.929306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:28.930231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:28.930231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:28.930231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:28.930231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:28.934506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 332/667] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:28.936152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:28.952498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:28.953544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:28.954106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:28.954531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:29.030051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:29.056225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:29.057772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:29.078288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba985d7-744d-4121-85f4-ae0e7c9e91eb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:29.134310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:29.136547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:29.137345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:29.138462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:29.157827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:29.167109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:29.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 333/668] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:29.178980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:29.211455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 336/669] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:29.212382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:29.248417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:29.357805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7d89d-dcd6-41e6-9bc7-7e20e1fbfe2f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:29.365751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:29.386213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af7f4b-7eee-49f0-b8b7-4072082fb19e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:29.409359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:29.410141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:29.410468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:29.410709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:29.411497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=c2f2db6b70c241ec8bda9c96330f98c7) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:29.411807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:29.413310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:29.420082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 334/670] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:29 2022] GET /identity/v3/role_assignments?user.id=c2f2db6b70c241ec8bda9c96330f98c7&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:29.427722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:29.438441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:29.439300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:29.439713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:29.440066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:29.440923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:29.441376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:29.442989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:29.460687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 337/671] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:29.462307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:29.648322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a539d20-e4ce-4630-bff5-da26d6325917 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:29.684608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fb787-54f2-420b-a83a-9336126882ae None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:29.691545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:29.694539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:29.695317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:29.695610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:29.695898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:29.709828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:29.711325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:29.719882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:29.745497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:29.747230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:29.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:29.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:29.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:29.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:29.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 338/672] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:29.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:29.769758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:29.771320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:29.771957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:29.772668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:29.774370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:29.775138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:29.783694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:29.802134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 335/673] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:29.804119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:29.946939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9983fd-0d0a-416a-a260-cf41998a88f7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:29.966094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd3a30-1123-4f40-9ba8-3bc0afb00910 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:30.008761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.009484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.009958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.010238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.019421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.021130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.025682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.026731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.027222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.027705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.029878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:30.030418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.033086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:30.035669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:30.037401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 339/674] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:30.040245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.073647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 336/675] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:30.073647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.175053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:30.181812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:30.223750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb7212-60dd-45e1-9571-e5e4838755e5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:30.259310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b4213-581f-4b12-99a5-d36247acd027 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:30.274540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.275279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.275629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.275976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.276945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:30.277396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.278987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.299876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 337/676] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:30.300573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.327923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.329061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.329551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.330033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.350880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.353195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-fb32c197-c777-4f7b-9754-5d6f3ef6d6a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:30.382114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:30.383753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:30.385582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:30.385582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:30.385582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:30.385582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:30.385582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 340/677] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:30.387556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234fd-6a97-4521-9b29-2e52c3a2d6f3 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.387556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234fd-6a97-4521-9b29-2e52c3a2d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.387556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234fd-6a97-4521-9b29-2e52c3a2d6f3 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.397060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 341/678] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:30.397060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234fd-6a97-4521-9b29-2e52c3a2d6f3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.436131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:30.457294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c0b1d-40f7-46fd-9fa5-1034c19655f4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:30.517901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.519164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.519687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.520352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.533799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.537892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:30.561901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:30.562993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 338/679] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:30.569686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.590487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234fd-6a97-4521-9b29-2e52c3a2d6f3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:30.607596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:30.644307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.645261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.645620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.646009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.646951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=a903f3771a62447d9c06fbeba414e9e3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:30.647347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.648993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8d759-1ede-4971-85d8-8f75f3dfa0ac None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.665001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 342/680] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/role_assignments?user.id=a903f3771a62447d9c06fbeba414e9e3&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:30.665001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.665001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.665001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.681756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:30.683252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:30.786497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631993c-a92c-4f55-8938-175124ff2a5d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:30.813608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 343/681] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:30.814825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.868005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:30.869711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:30.870727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:30.871308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:30.874650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:30.875332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:30.878751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:30.894381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 339/682] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:30.895386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:30.907270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:31.000325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6378c-7647-4f79-a5bc-62b4a6900ddd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.002472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:31.053261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.053812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.054104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.054374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.055169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:31.055710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.057773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.077361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 344/683] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:31.078126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:31.113473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923e32-a20a-48d4-a566-8a35917c156b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.193424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.194193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.194628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.195028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.208534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.210295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:31.228889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:31.230992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:31.233285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:31.233285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:31.233285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:31.233285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:31.237778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 340/684] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:31.238645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:31.253848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16813b7e-f055-4fd6-9430-1b00bdca96ba None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.315021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:31.344890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.346683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.348039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.348925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.373876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.377347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.382035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e0ea-e7fe-475b-90f3-e8693da763f4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:31.405624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:31.407539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:31.409531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:31.417104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 345/685] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:31.417104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:31.458556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:31.479547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.481558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.483053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.483869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.486178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=ed1fb62d8f3f4a73ac9d11a37c62f99f) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:31.487085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.490554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.505329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 341/686] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:31 2022] GET /identity/v3/role_assignments?user.id=ed1fb62d8f3f4a73ac9d11a37c62f99f&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:31.507605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:31.550807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256de90-fd43-4b73-8d3d-d7d8c399eda9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.593878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.594424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.594709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.594998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.595693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:31.595997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.597474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.616997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 346/687] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:31.630034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:31.738616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:31.764166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c852621-3aa6-4ecc-954b-f0aa85f26aad None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.778606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3155696c-ee69-4609-b453-bbe77d09ed7b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:31.806674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.807971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.807971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.807971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.808648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.810427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.831969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:31.837900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 347/688] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:58:31 2022] POST /identity/v3/users => generated 254 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:31.872939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:31.873237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:31.873494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:31.873772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:31.874689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:31.875004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:31.876549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0bf23-6702-4843-8305-c87b79de0da1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:31.900082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 342/689] 10.222.0.48 () {56 vars in 1135 bytes} [Tue Jul 26 16:58:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:31.906377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:32.126581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfc09-ed65-419e-bde9-7fa536a36c06 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:32.225589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:32.226731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:32.227201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:32.227706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:32.250964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:32.408940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:32.410494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:32.437190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8c411-c1fb-4d76-a62c-324133b8eb1d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:32.437727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8c411-c1fb-4d76-a62c-324133b8eb1d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:32.438156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8c411-c1fb-4d76-a62c-324133b8eb1d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:32.449910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 343/690] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:32 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:58:32.458233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:32.458742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:32.459247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:32.523236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:32.530258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:32.547072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 348/691] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:32.558797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:32.628306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 344/692] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:32.651693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:32.693613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:32.775732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf515ade-f447-4e56-89a9-431e7231df89 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:32.788891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:32.817856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:32.818528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:32.818808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:32.819072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:32.819717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:32.821158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:32.828732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8fb15-472b-4fdf-a9ba-d67cb12022c6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:32.851680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:32.853886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:32.855578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 345/693] 10.222.0.48 () {56 vars in 1129 bytes} [Tue Jul 26 16:58:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:32.867174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:32.892363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:32.893062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:32.893323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:32.893609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:32.917329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:32.921241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:32.947088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:32.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:32.954049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:32.954049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:32.954049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:32.954049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:32.954049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 349/694] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:32.960992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:33.097444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a511d-eb78-43a6-b22f-5d7862286f59 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:33.195899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:33.197182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.197612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.198146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.199428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:33.199986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:33.203253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb9323-2c76-46f0-a2d8-131355457885 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:33.218043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 346/695] 10.222.0.48 () {56 vars in 1124 bytes} [Tue Jul 26 16:58:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:33.221006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd0ed8-604a-48da-8d2a-469f764eb9f9 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.222210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd0ed8-604a-48da-8d2a-469f764eb9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.225961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd0ed8-604a-48da-8d2a-469f764eb9f9 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.231489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 347/696] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:33.233479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd0ed8-604a-48da-8d2a-469f764eb9f9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:33.274484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a8366-09f4-4da8-9dd9-015f266688a8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:33.281924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:33.287370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:33.330228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:33.331166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.331795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.332561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.334080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:33.334604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:33.336655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1e2ba-c637-47b8-9053-ac8e307b015a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:33.359312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 350/697] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:33.361140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.362277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.362780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.389742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:33.391256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:33.449604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd0ed8-604a-48da-8d2a-469f764eb9f9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:33.495979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 351/698] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:33.497271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:33.533190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:33.533839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.534118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.534388 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.535190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:33.535592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:33.537800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:33.547239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 348/699] 10.222.0.48 () {56 vars in 1124 bytes} [Tue Jul 26 16:58:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:33.548250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:33.713594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76d40a-9ba9-4909-bc62-c7fab0ebe126 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:33.718341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:33.749318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:33.760201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:33.761228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.761751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.762181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.775994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:33.784647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:33.785823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:33.786798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:33.790269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 349/700] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:33.797808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:33.837805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1d1aa-4929-497c-bd1e-c1c0b9748492 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:33.939127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:33.940724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:33.941473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:33.942385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:33.957118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:33.960408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:33.985407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:33.987352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:33.989877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:33.991492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 352/701] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:34.001144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:34.087053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fd4fc-4445-4620-97b4-ae35c6e88bbe None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:34.192642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:34.193756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:34.194235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:34.194742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:34.196119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:34.196672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:34.199441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:34.217691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 350/702] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:34.237237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:34.275853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b4d54-73b7-48a2-be6b-5855b24d19b5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:34.407118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:34.408088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:34.408511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:34.408921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:34.410182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:34.410641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:34.413090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:34.441603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:34.445911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 353/703] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:34.452749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:34.498305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:34.698993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4c09d-4b7c-4dd8-a121-8d93c4701d1d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:34.721396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfc514-6a2a-41f5-baae-352920be6f62 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:34.780479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:34.781454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:34.781930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:34.782409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:34.783903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=0bfb7deb3aaf48a6992edb1b01b17aff) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:34.784431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:34.787229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:34.800442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 354/704] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:34 2022] GET /identity/v3/role_assignments?user.id=0bfb7deb3aaf48a6992edb1b01b17aff&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:34.850432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:34.851453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:34.851913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:34.852371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:34.889653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:34.892132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c81d4879-c3f6-4c2f-bcb3-cbbfa0b1e2b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:34.928731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:34.930499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:34.932297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:34.932297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:34.932297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:34.932297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:34.932297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 351/705] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 695 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:34.939915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:35.090227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:35.097108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee9966-d693-40a7-a4dc-23a7821a03a3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.098002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee9966-d693-40a7-a4dc-23a7821a03a3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.098551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee9966-d693-40a7-a4dc-23a7821a03a3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.101274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 352/706] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:58:35.107925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.108560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.109018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.131132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:35.133188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:35.196577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddc41a-37b8-4a18-b50d-d0956bafbbce None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:35.247244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:35.248325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.248989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.249418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.250334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:35.250813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:35.252399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:35.256398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 353/707] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:58:35.277661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 355/708] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:35.284928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:35.289852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:35.295765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:35.464903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116d5cd-37c1-41ef-a694-5b46e1227c32 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:35.487619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a839b-d62b-49a0-899a-f77000f3925a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:35.525165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:35.540214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:35.541404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:35.541626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.542218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.542545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.542922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.543255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.543696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.553911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:35.555715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:35.567408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-bbe797e0-4326-47a3-8c74-7f465bc9634d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:35.571792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:35.572899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:35.573913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:35.576514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 356/709] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:58:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:35.577818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:35.578790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:35.579660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:35.581311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebca9-5f25-4be3-b719-d247f1f4745a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.581855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebca9-5f25-4be3-b719-d247f1f4745a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.582336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ebca9-5f25-4be3-b719-d247f1f4745a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.584699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 357/711] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:35.584863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 354/711] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:35.586511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:35.594864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.595503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.596039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.620086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:35.621948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:35.721126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:35.751977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 358/712] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:58:35.754173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:35.789365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894e55a-0241-4de8-81e0-7a368bbc42c9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:35.866174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:35.867602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:35.868090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:35.868585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:35.870211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:35.870731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:35.873522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:35.910451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 355/713] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:58:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:35.911987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:35.985573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67b375-a5c0-402a-aab2-fcbfcee9c707 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.014518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.033806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.034831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.035221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.036055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.037137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:36.037670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.039224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.051813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 359/714] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:36.052886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.114836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4048ff-a30e-4ab4-a05d-629dec7417e7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.159033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.159715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.159998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.160267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.161141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.162675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:36.173206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:36.175508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:36.177721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:36.180719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.185093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 356/715] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:35 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:36.185093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.208184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32a02c-09f1-496f-a96b-c93324b60570 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.259119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.260067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.260458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.260846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.308523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbfe0f-3e8f-49a0-b82b-e46e88c12ed8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.331598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.362064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.363182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.363689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.364228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.364899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.366446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.379492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.381727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:36.398961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:36.399940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:36.400845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:36.400845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:36.400845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:36.400845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:36.412242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 357/716] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:36.413491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.475271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.505268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 360/717] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:36.506745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.567305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a34d9d-cdd8-4f66-9c35-d01d4c7f2193 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.608378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.627943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.629550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.629876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.630149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.632312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] RBAC: Adding query filter params (name=network) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:36.632654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.634138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.644786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 358/718] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:36.646455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.662855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ce81-31df-48a6-9aa5-a9d574e4b82d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.729712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.730390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.730839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.731116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.732197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:36.732524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.734453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.755371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 361/719] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:36.756343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.832841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24435cbe-9e19-4a1d-9e47-525e8f370223 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.838090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.890090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.891433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.892182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.892743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.913337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.915887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.936453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42afc7-3791-476f-b224-b6a7b260a0f8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:36.946004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:36.947591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:36.951902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:36.951902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:36.951902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:36.951902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:36.955552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 359/720] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:36.956924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:36.985595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:36.986416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:36.986558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:36.987107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:36.987450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:36.987757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:36.989225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:36.994280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:36.999926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 362/721] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:37.000897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:37.137068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4562a7-861e-4250-a2cb-d11e5c2d56ee None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:37.191021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:37.192542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.193346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.193860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8a719-8bd2-4a15-a9b3-a8adc01618d7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:37.194526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.210853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:37.214358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:37.241000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:37.242900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:58:37.244744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:37.250885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:37.251564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.251882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 360/722] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:37.252045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.252323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.252886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:37.253209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:37.254646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51884a78-8d31-45d0-8574-258b4a95aa04 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:37.257671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:37.270580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 363/723] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:37.271950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50897f2-2ab7-44b3-8364-548a60da66b3 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.272258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50897f2-2ab7-44b3-8364-548a60da66b3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.272520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50897f2-2ab7-44b3-8364-548a60da66b3 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.274283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 364/724] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:37.282258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50897f2-2ab7-44b3-8364-548a60da66b3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:37.302726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:37.393980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:37.458742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808afc1-6cd2-467d-a5cb-b21ac0eb21e2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:37.491105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50897f2-2ab7-44b3-8364-548a60da66b3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:37.503074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:37.504119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.504617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.505131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.506168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:37.506774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:37.508616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1ede3-2c18-4c43-b628-bf4301231e1f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:37.535629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 361/725] 10.222.0.48 () {56 vars in 1149 bytes} [Tue Jul 26 16:58:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:37.537659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.538260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.538888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.561468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:37.563380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:37.565137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:37.566460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.566942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.567424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.582843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:37.585277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:37.608902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:37.609940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:37.610884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:37.617154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 365/726] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:37.618611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:37.658009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 362/727] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:37.660863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:37.665228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:37.781723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb79f8-cea9-4068-8f8d-1b33b3045cf8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:37.824081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:37.824911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.825324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.826423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.827113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b039d-9b84-4e49-b6ba-cc741c09ce0d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:37.827417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:37.827834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:37.829581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:37.845618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 363/728] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:37.847173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:37.867956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:37.899079 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481d85a-8ca3-4988-ad6b-edeb8c510c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:37.900491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481d85a-8ca3-4988-ad6b-edeb8c510c6b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:37.901413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481d85a-8ca3-4988-ad6b-edeb8c510c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:37.902163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481d85a-8ca3-4988-ad6b-edeb8c510c6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:37.948432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:38.030199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481d85a-8ca3-4988-ad6b-edeb8c510c6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:38.032025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481d85a-8ca3-4988-ad6b-edeb8c510c6b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:38.038752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793bd7b2-bbe7-4e07-babd-658a7dccb525 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:38.121395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:38.123415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:38.123927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:38.124374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:38.125431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:38.128114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:38.154640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 364/729] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:58:37 2022] POST /identity/v3/services => generated 251 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:38.154640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:38.181892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 366/730] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 16:58:38.184571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207afb-671e-46ec-9eb0-c5f591ac3a8c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:38.185494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207afb-671e-46ec-9eb0-c5f591ac3a8c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:38.186216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93207afb-671e-46ec-9eb0-c5f591ac3a8c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:38.189450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 367/731] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:38.200583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:38.201061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:38.201439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:38.245298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:38.249431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:38.282968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:38.396422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4015-bf3d-454a-ac66-9d355aa9b71a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:38.399812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 368/732] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 16:58:38.401810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:38.458773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:38.500712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:38.501820 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:38.502344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:38.502798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:38.504274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=a903f3771a62447d9c06fbeba414e9e3) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:38.505021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:38.507764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:38.521263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 365/733] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:38 2022] GET /identity/v3/role_assignments?user.id=a903f3771a62447d9c06fbeba414e9e3&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:38.522890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:38.619322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f89bd-5c1e-418a-bdfb-a3de7c313184 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:38.745982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:38.746532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:38.746821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:38.747089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:38.765124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:38.766692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:38.776462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df978e92-3651-46b9-b412-99fea0741c61 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:38.777339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:38.787166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:38.788178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:38.789137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:38.789137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:38.789137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:38.789137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:38.789137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 369/734] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:38.797165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:38.874220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:38.875404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:38.875827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:38.876185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:38.916794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:38.927343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:38.928375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:38.929264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:38.930010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 366/735] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:38.938105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:39.031673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a70acc-a9e4-4559-8bb2-a4e448e8c483 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:39.100649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:39.127079 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:39.129420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:39.129897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:39.130379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:39.131701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:39.132241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:39.137901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:39.177795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 370/736] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:39.187118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:39.287608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329434-b477-4fe5-870a-d270b417e05a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:39.326640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:39.398855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:39.399512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:39.399846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:39.400131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:39.400902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:39.401242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:39.402706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:39.413275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59d3b0-099e-4e0c-9100-1b30a731addb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:39.435481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 367/737] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:39.444251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:39.489216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:39.489895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:39.490176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:39.490456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:39.511884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:39.516298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:39.556672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:39.558717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:39.560458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 371/738] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:39.568437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:39.770847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e86485-4e2f-46d0-bc0e-00d351b37a6e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:39.799083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:39.832685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:39.833415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:39.833696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:39.833970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:39.846629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:39.848162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c2726a5d-f65c-4554-b330-be442f3c336f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:39.867333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:39.869075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 368/739] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71960d10-e1f5-4d32-849f-302cf392ebfc None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:39.870804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71960d10-e1f5-4d32-849f-302cf392ebfc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:39.871544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71960d10-e1f5-4d32-849f-302cf392ebfc None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:39.877201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 369/740] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:39.878322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71960d10-e1f5-4d32-849f-302cf392ebfc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:39.896758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581c19-0a02-4ba9-954c-e7a471e5e137 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:39.936177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:39.970790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:39.972086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:39.972700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:39.973347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:39.974883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:39.975573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:39.979419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c2685-0618-48b0-82cb-d937fda3a930 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:40.008634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71960d10-e1f5-4d32-849f-302cf392ebfc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:40.009334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 372/741] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:40.011921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:40.013890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:40.013890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:40.051107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:40.052576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:40.053232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:40.053946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:40.054443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:40.054921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:40.055892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:40.056443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:40.058247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:40.072918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 370/742] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:40.074400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:40.097842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:40.208785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 373/743] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 16:58:40.208785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:40.212044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1df264-757c-40b2-802f-7b306b9d4bfe None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:40.273904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:40.274962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:40.275458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:40.276020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:40.276866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=564ffb348a9b4554ac2cedb7eced0bb5) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:40.277224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:40.279241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:40.286903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 371/744] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:40 2022] GET /identity/v3/role_assignments?user.id=564ffb348a9b4554ac2cedb7eced0bb5&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:40.287584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:40.478583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54591bf4-1e48-4284-810e-5834f5ee2406 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:40.485727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:40.502081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:40.535839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0297077-0837-4653-8b8f-15706ad73544 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:40.566523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:40.567201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:40.567480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:40.567753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:40.579616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:40.581240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:40.604087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:40.605149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:40.606100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:40.609088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 374/745] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:40.612355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:40.621653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:40.622827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:40.623328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:40.623822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:40.624800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:40.628499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:40.647156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 372/746] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:58:40 2022] POST /identity/v3/services => generated 240 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:40.750073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202c341-18a1-4df3-b7d8-0b0acae87253 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:40.814826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:40.815926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:40.816417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:40.816949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:40.818047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:40.818570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:40.820313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:40.850262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 375/747] 10.222.0.48 () {56 vars in 1135 bytes} [Tue Jul 26 16:58:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:40.857178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:40.990384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cc4eb-be1e-4f49-b3d0-d10a447baea2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:41.033773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:41.034842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:41.035342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:41.035855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:41.051356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:41.127205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:41.129111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:41.252699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 373/748] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:58:41.258974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:41.290420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:41.392362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d99f1-22f2-4414-a153-9b289bb7d4ac None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:41.440069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:41.441156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:41.441668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:41.442145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:41.465971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:41.468159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ae10fc99-33d7-47ff-8517-8624d76d42b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:41.488939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:41.489924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:41.490828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:41.490828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:41.490828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:41.490828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:41.493824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 376/749] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:41.500881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:41.610543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:41.667484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c74d86-7b22-4779-bf71-5a204e437db6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:41.668156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c74d86-7b22-4779-bf71-5a204e437db6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:41.668717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c74d86-7b22-4779-bf71-5a204e437db6 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:41.671394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 377/750] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:41.677528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:41.678260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:41.678741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:41.703578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d01c03-5557-4167-88be-cbb50d8136fd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:41.714112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:41.720685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:41.758140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:41.759239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:41.759734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:41.760245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:41.761267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:41.761803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:41.763534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:41.778196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 374/751] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:41.785438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:41.879525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 378/752] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:58:41.920679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:41.932559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:41.992250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411db0c8-98ea-415a-96f9-ecac6cfcb551 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:41.994414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:42.062020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.063274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.063770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.064255 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.078734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:42.081872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:42.090650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212dae6-e273-4dab-92ee-44fab0380047 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3ef03db9-5f5a-47fe-82f5-323ebbf93717 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:42.106378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:42.108260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:42.109300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:42.111126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 375/753] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:42.112996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82a4b-4d53-490d-a8aa-40625238e284 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.112996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82a4b-4d53-490d-a8aa-40625238e284 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.113395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc82a4b-4d53-490d-a8aa-40625238e284 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.115902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 376/754] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:42.123030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82a4b-4d53-490d-a8aa-40625238e284 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:42.141125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5455-bac7-4394-936c-ad56e27a0d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.141715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5455-bac7-4394-936c-ad56e27a0d0e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.141992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5455-bac7-4394-936c-ad56e27a0d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.142264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5455-bac7-4394-936c-ad56e27a0d0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.155653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d5455-bac7-4394-936c-ad56e27a0d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9c2d5455-bac7-4394-936c-ad56e27a0d0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:42.161330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:42.163701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 379/755] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:42.165926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.167761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.167761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.195953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:42.197215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:42.269036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:42.285255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 380/756] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:42.286332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:42.351238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc82a4b-4d53-490d-a8aa-40625238e284 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:42.415809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.417150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.417676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.418183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.420346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:42.420938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:42.422957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:42.445523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 377/757] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:42.445523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:42.482429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:42.537859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e052456-4b90-4d6b-9049-6c67dcae664e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:42.593578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.594648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.595511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.596013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.598397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:42.599142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:42.602817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:42.622077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 381/758] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:42.624071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:42.631917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c62a89-c3ec-4031-9b6e-d0c9a68f201e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:42.711190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:42.722249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.723747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.724224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.724713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.725958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:42.730704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:42.754066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:42.755073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:42.755949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:42.761180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 378/759] 10.222.0.48 () {56 vars in 1129 bytes} [Tue Jul 26 16:58:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:42.764234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:42.790164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1b11-3736-4086-adb5-ef8a0c37d891 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:42.836241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.837375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:42.837883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:42.838352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:42.839434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=0bfb7deb3aaf48a6992edb1b01b17aff) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:42.839981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:42.842448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:42.857354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 382/760] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:42 2022] GET /identity/v3/role_assignments?user.id=0bfb7deb3aaf48a6992edb1b01b17aff&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:42.858815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:42.926184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:42.949122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6172039-2b6e-4de4-847f-fb1bed4bc84e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:42.998102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:42.999640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.000098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.000531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.017611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:43.019800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:43.040407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:43.041594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:43.042657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:43.042657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:43.042657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:43.042657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:43.050042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 379/761] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:43.052372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:43.147759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d9ec2-8ce2-4adf-99c9-a898ec5bc20f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:43.198572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:43.215465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b5f4bd-543c-4ce5-9fb3-1c19de3e3f4d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:43.233501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:43.234418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.234856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.235336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.236202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:43.239321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:43.241002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:43.255534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 383/762] 10.222.0.48 () {56 vars in 1124 bytes} [Tue Jul 26 16:58:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:43.264583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:43.293611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:43.295394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.295906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.296399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.297977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:43.298521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:43.301260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0a613-3034-419d-b217-3f4ceb866e09 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:43.315131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:43.324681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 380/763] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:43.331201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11ea03-4fc8-446b-b5ea-7524c8d7b6d6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.331201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11ea03-4fc8-446b-b5ea-7524c8d7b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.331201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11ea03-4fc8-446b-b5ea-7524c8d7b6d6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.331201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 381/764] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:43.333548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11ea03-4fc8-446b-b5ea-7524c8d7b6d6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:43.488274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbc08b-43b9-493c-83cb-8549868c9712 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:43.566481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11ea03-4fc8-446b-b5ea-7524c8d7b6d6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:43.570007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:43.572897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.573804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.574566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.576621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:43.577433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:43.580113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add22f1-293b-4232-9c23-2b767b38ab20 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:43.593676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 384/765] 10.222.0.48 () {56 vars in 1124 bytes} [Tue Jul 26 16:58:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:43.596537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.597424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.598193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.635366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:43.637827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:43.653398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:43.655889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:43.656824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:43.657704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:43.673359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:43.676637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:43.701779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:43.703625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:43.707342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:43.711421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 382/766] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:58:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:43.713295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:43.737354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:43.795970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 385/767] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:43.797706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:43.845412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:43.934555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a12aff-a776-4b90-a7a6-1a31cf1345f6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:44.008173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:44.010471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.011124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.012137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.013940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] RBAC: Adding query filter params (interface=public, service_id=fb6fc60a21a74796a5c40f4fc05068f2, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:44.014700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:44.018505 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:44.041094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 383/768] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:58:43 2022] GET /identity/v3/endpoints?service_id=fb6fc60a21a74796a5c40f4fc05068f2&interface=public®ion_id=RegionOne => generated 194 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:44.042818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:44.056577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea19938-4940-4f5b-94a8-cf0a145e0f08 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:44.135005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:44.136239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.136875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.137519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.139860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:44.140551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:44.144512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:44.192848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 386/769] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:58:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:44.194416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:44.272854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee7f30-840b-4396-b7f7-e0f56387d1c9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:44.273666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:44.319852 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:44.342368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:44.343100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.343419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.343945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.364353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:44.369946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:44.371797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 384/770] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:44.374669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:44.392648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a5d19-54b2-41cf-8639-2ee906173cbc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:44.463704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f76cf7-1015-482f-b1bd-397f75ed2713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:44.464764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f76cf7-1015-482f-b1bd-397f75ed2713 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.465297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f76cf7-1015-482f-b1bd-397f75ed2713 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.465865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f76cf7-1015-482f-b1bd-397f75ed2713 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.467589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f76cf7-1015-482f-b1bd-397f75ed2713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:44.470243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f76cf7-1015-482f-b1bd-397f75ed2713 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:44.488753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 387/771] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:58:44 2022] GET /identity/v3/services => generated 772 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:44.488753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859749-3e29-44e2-ae7d-53e1f6b29bbc None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.488753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859749-3e29-44e2-ae7d-53e1f6b29bbc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.488753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859749-3e29-44e2-ae7d-53e1f6b29bbc None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.488753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 388/772] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:44.488753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859749-3e29-44e2-ae7d-53e1f6b29bbc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:44.608360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2ab76-b827-4025-8072-5c234e60eabc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:44.693759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:44.695042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.695668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.696249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.707842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:44.713414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859749-3e29-44e2-ae7d-53e1f6b29bbc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:44.732334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:44.735237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:44.755832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:44.757081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.757422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.757797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.758642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:44.759056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:44.760658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6082f-02a6-47f3-b9b9-01e85f631128 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:44.772199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 389/773] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:44.773782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:44.774181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:44.774551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:44.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:44.776792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:44.780423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:44.780423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:44.780423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:44.780423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:44.782779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 385/774] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:44.788016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:44.807632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:44.809439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:44.859562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:44.944206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 390/775] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:44.944206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.041091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53867e1-7208-4509-9bf0-1ff369ba0a77 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.112025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:45.113619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.114378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.115113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.126342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:45.139270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:45.142461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:45.142985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e77950-5e4c-463f-bc53-6a63e62d1f36 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:45.171449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:45.173203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:45.174960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:45.174960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:45.174960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:45.174960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:45.179196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 386/776] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:45.181195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.209855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:45.210771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.211244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.211714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.212977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:45.213522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:45.216709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:45.244998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 391/777] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:45.244998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.359339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:45.390439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21b20e-6ead-49a7-aba6-6fee948bdb9a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.428792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf66816-6a2f-457a-89c3-26322b8ad073 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.453873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:45.454544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.455501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.455790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] PATH_INFO: `/v3/services/network` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.456456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:45.457934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:45.468693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:45.478730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:45.480692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 387/778] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:45 2022] GET /identity/v3/services/network => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:45.483645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.490455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:45.491547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.491998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.492592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.493866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:45.494412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:45.496849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5fca2b-e51d-4226-8bcd-7b98d0765dc4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:45.514047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 392/779] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:45.515617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090db08-73b6-4d99-a34a-33a7a7ff7eb1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.516067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090db08-73b6-4d99-a34a-33a7a7ff7eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.516523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090db08-73b6-4d99-a34a-33a7a7ff7eb1 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.521250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 393/780] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:45.521250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090db08-73b6-4d99-a34a-33a7a7ff7eb1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.542036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:45.636238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae60876-2ad5-4e83-b810-9e348f85da9f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.687059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090db08-73b6-4d99-a34a-33a7a7ff7eb1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.703492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:45.704629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.705160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.705642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.720538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:45.723216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:45.737548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:45.738421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.738827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.739276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.740126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] RBAC: Adding query filter params (name=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:45.740557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:45.742242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1a258be5-0e49-4f76-af96-f528d2213354 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:45.748316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:45.750744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:45.754401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:45.756181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 394/781] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:58:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:45.756181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.762780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 388/782] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:45.762780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:45.762780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:45.762780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:45.792682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:45.794915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:45.800608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:45.881058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:45.925240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 389/783] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:45.925240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:45.930554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf856595-e074-4863-a2d2-1fdcd0780a17 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:45.999264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.000275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.000724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.001253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.017914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.021569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:46.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:46.045148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:46.047250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 395/784] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 16:58:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:46.050040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.092211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47531604-437a-4c71-90fd-1d10699eee2b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.139936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.140940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.141508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.141940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.142790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:46.143268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.145274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.159817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 390/785] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:58:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:46.159817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.162105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:46.258502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d1211-a690-46da-bde3-0ec065fe8534 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.314869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.315987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.316344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.316689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.317867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:46.318375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.320854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.327521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:46.342638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 396/786] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:46.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcfc75-933e-47f9-a116-a79527cb3de6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.344409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.426024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.427124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.427581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.427993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.442824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:46.447969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:46.449017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:46.449895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 391/787] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:46.451166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.508493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aec04-7e91-4b8a-a162-555ab94c5fe4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.553751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:46.561015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.562040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.562450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.562867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.564010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:46.564384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.566143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.599737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 397/788] 10.222.0.48 () {56 vars in 1149 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:46.599737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.620707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fd75e-fc44-45ae-8287-63129e7840c4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.667195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.668196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.668617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.669123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.670140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] RBAC: Adding query filter params (interface=public, service_id=5a678d0a32b841c3857635068309c379, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:46.670673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.672484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.682811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 392/789] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/endpoints?service_id=5a678d0a32b841c3857635068309c379&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:46.683072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:46.683216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.747373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54a34-1319-4207-894c-bd9f61ed922b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.794780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.795746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.796202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.796475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/a903f3771a62447d9c06fbeba414e9e3/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.798742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=a903f3771a62447d9c06fbeba414e9e3, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.800317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.822141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 398/790] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:46 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/a903f3771a62447d9c06fbeba414e9e3/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:58:46.823244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:46.837902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7596af6-28f8-44cd-a39b-b42f5ea6d395 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:46.906170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:46.922250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:46.924235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:46.925134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:46.926104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:46.928147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:46.930146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:46.934012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:46.962085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 393/791] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:46.962986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.020920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c218dd-c0ab-4ca2-af42-4fb7c5a9dbf3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.046534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:47.090853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.091990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.092508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.092940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.110505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.112626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:47.134049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:47.135161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:47.136199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:47.136199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:47.136199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:47.136199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:47.143179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 399/792] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:47.143179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.195496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4671e-c58f-467b-b3db-1a68bc902ed0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.262271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.264057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.264550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.265086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.266303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.268924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.288994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 394/793] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:58:46 2022] GET /identity/v3/services => generated 772 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:47.288994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.294662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf2754-6e80-497f-aa0b-6067a2ea4c2c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.346685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.347490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.348040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.351243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.367774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.369690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.398399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e67f304e-d296-4625-8255-28cdad3b971f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:47.404993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:47.406328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:47.407358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:47.407358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:47.407358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:47.407358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:47.407358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 400/794] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:47.408425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6265f-3858-460a-86bd-ea9c2ce67367 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.409293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6265f-3858-460a-86bd-ea9c2ce67367 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.409758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6265f-3858-460a-86bd-ea9c2ce67367 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.414855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 401/795] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:47.415055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6265f-3858-460a-86bd-ea9c2ce67367 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.455124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:47.500578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a66c1-ec3b-45a7-8470-1d45e20ed5c0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.563276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.563913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.564190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.564462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.565219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:47.565528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.566952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14aac85-bda5-4d85-8888-39a69ccc3756 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.581736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 395/796] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:47.583101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.583385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.583659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.607014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:47.608265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:47.610223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6265f-3858-460a-86bd-ea9c2ce67367 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.677616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.678810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.679295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.679755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.681054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:47.681589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.684471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.700202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 396/797] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:47.706346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.715828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 402/798] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:47.729357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.789714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:47.840920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9190f9-c783-4315-9dc2-b708ef9f4abf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.853784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:47.880716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70c754-8f85-407a-abfe-4f3e632775f1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:47.883925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.884900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.885568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.886054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.894962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.897047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:47.912518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:47.913558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:47.914559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:47.922593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 397/799] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:47.923220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:47.923872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:47.924150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:47.924420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:47.924890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:47.934318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:47.937084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:47.954505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:47.955590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:47.957263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:47.960690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 403/800] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:47.961915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.052182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c4466-af82-4b11-9bf4-a890a213100b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.068640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:48.098210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.099585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.100138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.100685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.101746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.103526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:48.117796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:48.118867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 16:58:48.119740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:48.128098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 398/801] 10.222.0.48 () {56 vars in 1129 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:48.129441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.249844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409cb54-0030-4150-9316-278e51ef07c8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.261436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07126cf6-c71d-47fc-a22e-2755b11d25c6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.318685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.319321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.319606 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.319884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.320678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:48.321112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.322518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.329172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.329832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.330117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.330402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.331260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:48.331576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.333124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.339759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:48.369516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 399/802] 10.222.0.48 () {56 vars in 1135 bytes} [Tue Jul 26 16:58:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:48.369516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.375856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 404/803] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:48.376858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.434954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:48.505547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1803-3ec7-4c25-ae9a-de2fa1f0850b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.549440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.550461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.550945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.551425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.565041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.566863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:48.584403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:48.585433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:48.586344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:48.586344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:48.586344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:48.586344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:48.588866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 405/804] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:48.590126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.606276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4577678-b12d-4917-9b87-bb2817aa05ec None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.655645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.656638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.657191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.657656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.658544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:48.659038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.660672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.672105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 400/805] 10.222.0.48 () {56 vars in 1124 bytes} [Tue Jul 26 16:58:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:48.673484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.706788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:48.788514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679f2a-07e2-446e-bd11-858b6bc545eb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.849229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.851167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.851706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.852060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/564ffb348a9b4554ac2cedb7eced0bb5/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.859133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:48.873604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1aaedf-7793-4e9a-9013-973ff106b8d3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:48.882780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=564ffb348a9b4554ac2cedb7eced0bb5, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.884566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.909352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 406/806] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:48 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/564ffb348a9b4554ac2cedb7eced0bb5/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:58:48.910677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:48.946254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:48.947805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:48.948394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:48.949165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:48.951273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:48.951880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:48.954790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:48.976499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 401/807] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:48.987464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:49.124631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:49.127954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ffbc1-305c-4631-bd87-24acbf548a45 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:49.241343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3244b5-977e-4262-a236-b1e55b6d101b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:49.258209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:49.259166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:49.259634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:49.260122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:49.261035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:49.261537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:49.263261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:49.273180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 407/808] 10.222.0.48 () {56 vars in 1124 bytes} [Tue Jul 26 16:58:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:49.277326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:49.329795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:49.331025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:49.331843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:49.332423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:49.355350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:49.357982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:49.372213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:49.374267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:49.375233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:49.377953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 402/809] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:49.393368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:49.425421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:49.498761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b8de0-1d6a-4d94-9294-ddeed8c211b6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:49.566259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:49.568152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:49.571145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:49.572091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:49.576353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:49.579566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:49.661025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 408/810] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:58:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:49.664946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2fb64-2761-46f9-b53d-ed752f06c033 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:49.711100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:49.711744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:49.712024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:49.712294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:49.713094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:49.713400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:49.714875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6824162-6223-4047-983f-bbe0fd646bfd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:49.730305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 403/811] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:49.737488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:49.765730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:49.973104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37da11-8ca7-47df-b0b7-2142aed663d4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:50.052664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9cbf0-15f9-4c1f-bb01-a556c94dea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:50.054489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9cbf0-15f9-4c1f-bb01-a556c94dea49 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:50.055335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9cbf0-15f9-4c1f-bb01-a556c94dea49 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:50.056130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9cbf0-15f9-4c1f-bb01-a556c94dea49 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/0bfb7deb3aaf48a6992edb1b01b17aff/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:50.060580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9cbf0-15f9-4c1f-bb01-a556c94dea49 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=0bfb7deb3aaf48a6992edb1b01b17aff, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:50.063880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9cbf0-15f9-4c1f-bb01-a556c94dea49 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:50.095833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 409/812] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:58:49 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/0bfb7deb3aaf48a6992edb1b01b17aff/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:58:50.152785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39404d15-b899-45dc-a5aa-b1f95477282f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:50.153228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39404d15-b899-45dc-a5aa-b1f95477282f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:50.153390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39404d15-b899-45dc-a5aa-b1f95477282f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:50.155523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 404/813] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:50.164559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefe9f3-43c0-43e3-b7e1-038bb26034b0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:50.165049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefe9f3-43c0-43e3-b7e1-038bb26034b0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:50.165386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefe9f3-43c0-43e3-b7e1-038bb26034b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:50.189094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0cefe9f3-43c0-43e3-b7e1-038bb26034b0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:50.190439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0cefe9f3-43c0-43e3-b7e1-038bb26034b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:50.327042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 410/814] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:50.369599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39404d15-b899-45dc-a5aa-b1f95477282f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:50.418774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a26ce0-8716-441d-b611-ff3bcff72e47 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:50.419954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a26ce0-8716-441d-b611-ff3bcff72e47 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:50.420529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a26ce0-8716-441d-b611-ff3bcff72e47 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:50.423798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 411/815] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:50.433156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:50.433790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:50.434400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:50.480630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:50.487314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:50.489531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:50.495286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:50.572167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39404d15-b899-45dc-a5aa-b1f95477282f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:50.634404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 412/816] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:50.662580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:50.663264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:50.663548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:50.663823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:50.666576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:50.682968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:50.689233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:50.691027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:50.692142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:50.693178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 405/817] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:50.701953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:50.936449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a86a1-3541-4c93-8456-85a059f97406 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:50.962551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28167003-8c4b-48d0-8106-f3a24cb7be76 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:51.024091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:51.044018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:51.045836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:51.046656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:51.047382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:51.048614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:51.049435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:51.051642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:51.052390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:51.052676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:51.052825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:51.052987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] PATH_INFO: `/v3/services/network` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:51.053690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:51.055659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:51.072686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 406/818] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:51.084391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:51.086426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:51.088099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 413/819] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:50 2022] GET /identity/v3/services/network => generated 88 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:51.100556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:51.114089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:51.131072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:51.367300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537194c-2de9-469e-b858-a1bbe945c1be None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:51.451646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff25226-9e27-4bed-8b30-2a27b750c125 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:51.465893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:51.466759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:51.467053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:51.467192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:51.480695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:51.482361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:51.512279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:51.515131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:51.516935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:51.516935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:51.516935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:51.516935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:51.525061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 407/820] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:51.526217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:51.527044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:51.527541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:51.528104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:51.529467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] RBAC: Adding query filter params (name=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:51.530031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:51.530306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:51.533089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:51.543607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 414/821] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:58:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:51.554472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:51.635802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:51.695422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:51.768709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548d6a8-f9d0-4b4e-8268-540fc6cf331c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:51.811116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374cca7-eb56-4223-adc2-873452c6f981 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:51.836547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:51.837703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:51.838200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:51.838712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:51.840012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] RBAC: Adding query filter params (type=network) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:51.840549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:51.844463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:51.855102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 415/822] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:58:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:51.865577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:51.900727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:51.901296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:51.901568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:51.901822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:51.902550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:51.902850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:51.904453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:51.931635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 408/823] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:51.937734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:52.075362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c770f6-3af7-4f5d-b813-52445884a95b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:52.101447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:52.112101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea8fc3-3a6d-4c75-9227-5abccd92be7e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:52.140044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:52.150543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:52.151519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.151877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.152330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.154346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:52.157082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:52.176805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:52.177365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.177681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.177936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.190760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:52.192160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:52.207645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 416/824] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:58:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0e69b6a3-5bd4-44f4-b43c-d80d6b10f47c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:52.214749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:52.219369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:52.227784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 409/825] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:58:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:52.230671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:52.259028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee3dd9-b7af-4929-8645-7ea23706ef35 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.259518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee3dd9-b7af-4929-8645-7ea23706ef35 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.259518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee3dd9-b7af-4929-8645-7ea23706ef35 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.261587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 410/826] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:58:52.268763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.269203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.269498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.297144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:52.298360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:52.478103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfaf7c6-de74-451d-8b1f-307993c81801 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:52.484256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 411/827] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:58:52.507643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:52.566106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:52.567746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.568257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.568758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.570236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:52.570753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:52.574627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:52.602369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 417/828] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:58:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:52.610280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:52.661045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:52.673180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:52.692738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c3074-a865-4f25-9607-9d0e25631d06 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:52.749371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:52.750466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.750993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.751644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.765201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:52.770863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:52.771977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 16:58:52.772871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:52.773601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860adc6-7f82-4c39-8536-6fc91c55c6bf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:52.781940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 412/829] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 16:58:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:52.783332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:52.840860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37897f5-c7be-4fce-8b7b-8b39427dd6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:52.842785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37897f5-c7be-4fce-8b7b-8b39427dd6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.843270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37897f5-c7be-4fce-8b7b-8b39427dd6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.843735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37897f5-c7be-4fce-8b7b-8b39427dd6a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.906404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb42718-0ead-459d-8d77-1a98ffa6242b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:52.948386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:52.948919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.949249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.949520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.950242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:52.951110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:52.952552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14654-b8ae-4678-aed6-57fce8009101 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:52.969012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 413/830] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 16:58:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:52.969012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cdef5-0f0d-4989-94ac-faa74db2eb32 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:52.969012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cdef5-0f0d-4989-94ac-faa74db2eb32 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:52.969012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cdef5-0f0d-4989-94ac-faa74db2eb32 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:52.969012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 414/831] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:52.975315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cdef5-0f0d-4989-94ac-faa74db2eb32 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:52.983607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37897f5-c7be-4fce-8b7b-8b39427dd6a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:52.986518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37897f5-c7be-4fce-8b7b-8b39427dd6a7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:52.992424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:53.068063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:53.147120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cdef5-0f0d-4989-94ac-faa74db2eb32 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:53.183102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 418/832] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 16:58:53.183102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.183102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.183102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.213574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:53.215414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:53.217881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:53.218978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.219456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.219930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.240710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:53.243486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:53.272647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:53.274639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:53.276333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:53.276333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:53.276333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:53.276333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:53.276333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 415/833] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:53.276875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:53.341009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 419/834] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:53.341009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:53.421098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:53.479832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3e03-9358-4e39-9243-5ad95ce5055f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:53.488706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:53.530604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:53.531890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.532286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.532630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.536345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e65cc-4abb-4d83-bf73-0482fc2ca70a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:53.548658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:53.550592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:53.568974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:53.570861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:53.572649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:53.572649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:53.572649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:53.572649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:53.572649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 416/835] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:53.572649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:53.582218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:53.583495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.584014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.584472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.585689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:53.586194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:53.588812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac0175-ed46-4ef6-83b7-7fd06606150e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:53.605506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 420/836] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:53.609240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5628-76d5-48e9-9ae6-482e51bf970e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.609736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5628-76d5-48e9-9ae6-482e51bf970e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.610222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5628-76d5-48e9-9ae6-482e51bf970e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.613040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 421/837] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:53.614275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5628-76d5-48e9-9ae6-482e51bf970e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:53.720399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac3482-e258-4f35-9c77-6abaed93ed90 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:53.741264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5628-76d5-48e9-9ae6-482e51bf970e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:53.741537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:53.773087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:53.774038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.774360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.774772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.775585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:53.777176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:53.787132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:53.789838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:53.792241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:53.795153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 417/838] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:58:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:53.798221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:53.798461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:53.798946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.799553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.800114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.801916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:53.805461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:53.807102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e87d3a-e2b6-4692-a491-bb7267918326 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:53.821260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 422/839] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:53.822592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:53.822901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:53.823151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:53.848344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:53.849961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:53.967449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9234646-0101-4d07-b575-4c254421f11f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:53.985005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 423/840] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:53.985005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.031096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.031657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.031938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.032212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.041512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.043010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:54.056365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:54.057585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 418/841] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 16:58:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:54.059468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.114434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:54.133777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448895a1-d508-4ab1-81e2-ed787341f4c0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.168764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:54.185852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.187090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.187090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.187389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.188082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:54.188335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.189757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.195201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 424/842] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:58:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:54.196636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.212570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58fe1a-3b9d-4b66-bef9-96ef7e6a7f32 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.262599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.264100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.264593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.265091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.275254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.277911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:54.293903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:54.296552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 419/843] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:54.298876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.350482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db354d6f-e07a-462d-9fa2-7098ebd08fee None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.392003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.392682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.392980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.393267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.407261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:58:54.412910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:54.414885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 425/844] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:54.416855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.431613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:54.438358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495aef9-8603-4862-821a-1e81d9973487 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.476456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:54.479514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.480223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.480567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.480847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.481728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:54.482579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.484266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.504940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 420/845] 10.222.0.48 () {56 vars in 1149 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:54.505843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.545662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed172ca9-db3f-4fba-ad74-8bbe9149cc79 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.586509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.587154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.587437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.587710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.588403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:54.588704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.590194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.596521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 426/846] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:54.597421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.642687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b768c60-820c-4e36-a7bd-dff1a7fb3efd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.692899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.694103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.694686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.695135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.696120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:54.696625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.698394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.717203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 421/847] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:54.718741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.746091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:54.755675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2566c1-adc3-4fbe-99bb-b2abd7ac401a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.773926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:54.816590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.817242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.817735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.818016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.819773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:54.820086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:54.821612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:54.835567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 427/848] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:54.836999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:54.856283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e2df2-9b3a-4761-b51a-e030a5c58813 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:54.903410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:54.904508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:54.906062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:54.906062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:54.980704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacf556-00fe-4b3d-bacc-020520bf7354 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.001380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.003181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.022642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:55.023709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:55.024150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:55.024723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:55.025264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:55.026146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=a903f3771a62447d9c06fbeba414e9e3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:55.026495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.028068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.035832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 428/849] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/role_assignments?user.id=a903f3771a62447d9c06fbeba414e9e3&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:55.037065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.094524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:55.181510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 422/850] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:58:55.185594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.236338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877bf18-9428-4513-81b7-8d23997c78e9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.307748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:55.308789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:55.309283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:55.309731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:55.333622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.335792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:55.365929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:55.366907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:55.367887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:55.367887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:55.367887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:55.367887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:55.367887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 429/851] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:55.374966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.394071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49455992-7fb1-44f3-ab1c-81e579195ee9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.422745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:55.465123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:55.466405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:55.467013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:55.467507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:55.490077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.493281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.498087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:55.512318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b92b6d-5dac-4cd4-8231-8822d2cddf1a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:55.529510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:55.531291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:55.533650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:55.533650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:55.533650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:55.533650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:55.537288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 423/852] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:58:55.542799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.558175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:55.558915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:55.559272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:55.559645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:55.561898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:55.562212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.563647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.587750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 430/853] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:55.588661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.689344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee0bad-40b6-412b-aa19-4c595e81ba43 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.704887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a90844-5e1d-4f61-8c8d-a5b2e3e17cd5 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.735847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:55.737123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:55.737744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:55.738245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:55.739202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:55.739732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.741657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.746358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:55.747029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:55.747305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:55.748124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:55.756769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:55.758269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:55.765152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 424/854] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:55.766641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:58:55.772467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:55.773556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:55.774525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 431/855] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:55.780155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:55.794687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:55.808544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:55.950504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f9510-9b0a-4871-85b1-0228b74b1118 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:55.967592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9e7b4-cdda-4868-ae65-97ce5a3b39ce None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:56.034959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:56.036149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:56.036655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:56.037182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:56.038782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:56.039107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:56.041130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:56.042772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:56.044931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:56.045780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:56.046530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:56.060480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:56.063406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:56.081550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 432/856] 10.222.0.48 () {56 vars in 1135 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:56.083831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:56.085554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:56.086970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:56.090809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:56.093234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 425/857] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:56.096004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:56.243279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:56.289618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:56.321342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d1fa4-f772-4ae1-b117-a5bd66dc76a6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:56.323963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a49e69-120e-486a-9ea4-1ac0f4e94358 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:56.400184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:56.401150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:56.401562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:56.402070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:56.404267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:56.406802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:56.407203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:56.407405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:56.408185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:56.408984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:56.410275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:56.437493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 426/858] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:56.442267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:56.545870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:56.548715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:56.619782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172f2d-7ea5-4603-bdec-e428d6e342e3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:56.675438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:56.676488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:56.677009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:56.677524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:56.679068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=564ffb348a9b4554ac2cedb7eced0bb5) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:56.679629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:56.682516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:56.700338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:56.701346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 427/859] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:56 2022] GET /identity/v3/role_assignments?user.id=564ffb348a9b4554ac2cedb7eced0bb5&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:58:56.804840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 433/860] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:58:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:58:56.807240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:56.833597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:57.025481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b97def-a579-4564-87d7-ac774bf916ed None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:57.097997 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:57.099116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.099712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.100074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.124917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:57.128073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:58:57.156813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:57.158824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:57.160860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:58:57.160860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:58:57.160860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:58:57.160860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:57.164178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 428/861] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:57.174345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:57.375022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56b8f4-e11c-4f35-8944-3ada547cf925 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:57.421850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:57.422495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.422828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.423102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.423821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:57.424116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:57.425787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0c236-0289-4761-a4fa-8fcaf1c0ff75 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:57.440110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 434/862] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:57.448828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:57.548014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19bc8f-af7c-417c-ba38-9f4ad695d3f1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.548448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19bc8f-af7c-417c-ba38-9f4ad695d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.548905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19bc8f-af7c-417c-ba38-9f4ad695d3f1 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.552758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 435/863] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:58:57.560111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.560574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.563783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.584015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:57.597541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:57.599526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:57.643141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97d298-d1ba-4f54-bdef-cd856fc2724c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:57.651170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:57.701903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41af109-6901-4997-992d-121066ac584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:57.702777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41af109-6901-4997-992d-121066ac584f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.703067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41af109-6901-4997-992d-121066ac584f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.703345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41af109-6901-4997-992d-121066ac584f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.715878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41af109-6901-4997-992d-121066ac584f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:57.718126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41af109-6901-4997-992d-121066ac584f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b41af109-6901-4997-992d-121066ac584f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:58:57.731371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:57.732499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:58:57.733433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 429/864] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:57.745013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41af109-6901-4997-992d-121066ac584f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:57.745231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 436/865] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:58:57.789451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:57.907360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203264e-73cf-40c6-bb3a-87ac0deb114c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:57.917781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41af109-6901-4997-992d-121066ac584f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:57.948722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48551432-2931-4f82-8857-5912546796e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:57.949813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48551432-2931-4f82-8857-5912546796e0 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.950342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48551432-2931-4f82-8857-5912546796e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.951402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48551432-2931-4f82-8857-5912546796e0 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.952366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48551432-2931-4f82-8857-5912546796e0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:57.954386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48551432-2931-4f82-8857-5912546796e0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:57.968761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:57.979223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 437/866] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:58:57 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:58:57.981394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d6e42-3549-41f0-a39a-359fcdb7128e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.981873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d6e42-3549-41f0-a39a-359fcdb7128e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.982322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d6e42-3549-41f0-a39a-359fcdb7128e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:57.984472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 438/867] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:57.990316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:57.990672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:57.990862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:58.006077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:58.007963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:58.008830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:58.009360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:58.010382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:58.010927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:58.012785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:58.012998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:58.014964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:58.031791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:58.034890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 430/868] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:58.042334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:58.133457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 439/869] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:58:58.149384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:58.193194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cdf71-8e05-423d-af99-13274c31fd58 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:58.237133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:58.237784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:58.238065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:58.238407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:58.239234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=0bfb7deb3aaf48a6992edb1b01b17aff) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:58.239583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:58.241110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:58.253342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 431/870] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:58:58 2022] GET /identity/v3/role_assignments?user.id=0bfb7deb3aaf48a6992edb1b01b17aff&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:58.278142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31a97d-1cd3-401b-b219-e3524f97f840 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:58.325788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:58.327129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:58.327775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:58.328198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:58.329074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:58.330707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:58:58.342424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:58:58.343749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:58:58.344846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:58:58.347729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 440/871] 10.222.0.48 () {56 vars in 1132 bytes} [Tue Jul 26 16:58:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:58:58.358279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:58.442173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:58.496870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d15b-0b8f-4486-8298-4bd5add12ebd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:58.536603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:58.537279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:58.537561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:58.537830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:58.538534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:58.538833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:58.540236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf33e12-996b-4e28-88da-a6956f7d1a9c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:58.546870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 432/872] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:58:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:58.556268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:58.581905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:58:58.766495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac37092-f4bf-470a-b748-d59f56e1e3b7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:58.835907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:58:58.837041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:58.837482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:58.837888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:58.839094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:58:58.839645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:58:58.842546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53581a-5a3e-454e-a72f-eee8d620b12e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:58:58.852379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 441/873] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:58:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:58:59.696298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d03c6-94c6-4441-aed1-a4e66b8c6f9b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:59.696822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d03c6-94c6-4441-aed1-a4e66b8c6f9b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:59.696822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d03c6-94c6-4441-aed1-a4e66b8c6f9b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:59.698853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 433/874] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:58:59.705368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:58:59.705884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:58:59.705991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:58:59.726356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:58:59.727577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:58:59.824833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 442/875] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:58:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:58:59.848234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d03c6-94c6-4441-aed1-a4e66b8c6f9b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:58:59.974691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d03c6-94c6-4441-aed1-a4e66b8c6f9b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:58:59.984358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:00.020362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:00.021507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.021996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.022473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.035781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:00.041291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:00.042333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:00.043198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:00.046750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 434/876] 10.222.0.48 () {56 vars in 1120 bytes} [Tue Jul 26 16:58:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:00.057073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:00.057073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:00.243946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418feff5-8a52-4c2a-93e8-7781854cc628 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:00.313534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:00.314651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.315173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.315671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.317099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:00.317662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:00.320327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f78fd-1c1e-4cfb-950b-f3bc99759ae1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:00.337931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 443/877] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 16:59:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:00.344053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:00.480260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464a953-f8d0-427b-b471-f438d3ae372b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:00.508043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4871a-b163-4599-8675-a5f71294c7de None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.508549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4871a-b163-4599-8675-a5f71294c7de None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.509019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4871a-b163-4599-8675-a5f71294c7de None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.515380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 444/878] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:00.517554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.518109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.518632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.520908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:00.521607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.521889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.522168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.536187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:00.537990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:00.547441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:00.549683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:00.554926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:00.555990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:00.557011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:00.557011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:00.557011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:00.557011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:00.558896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 435/879] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:00.565086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:00.587847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:00.639673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:00.655383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 445/880] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:00.675669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:00.709224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a3616-ad79-4ecf-8bf2-2e9bd76d4b2e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:00.760535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:00.761301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.762169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.762553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.763712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:00.764126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:00.766014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:00.785137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 436/881] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:00.791104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:00.818809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf852d-7f3b-4dee-9f55-302d7a6fc728 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:00.861504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:00.862444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.862873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.863293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.864242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:00.865990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:00.877414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:00.878480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:00.879328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:00.882351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 446/882] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:00.895053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:00.903232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:00.940119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be352e-b9d7-480f-a5da-ce9d82d95395 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:00.983592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:00.984245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:00.984523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:00.984890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:00.993952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:00.995502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:01.009608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:01.011531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:59:01.013915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:01.015704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 437/883] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 16:59:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:01.016812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:01.060393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:01.135062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189fb6c-7a5d-4d91-b253-8190a6d5c3cb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:01.183112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea1512-4610-423f-83d1-fc17fce77b54 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:01.212697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:01.213438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.214359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.214359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.214811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:01.215081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:01.216700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:01.228997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 447/884] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:01.234630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:01.254416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:01.255232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.255608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.255996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.256926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:01.257998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:01.258954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a67fb4-0870-489f-85fc-4ddc417a1d23 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:01.271484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:01.290084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 438/885] 10.222.0.48 () {56 vars in 1149 bytes} [Tue Jul 26 16:59:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:01.293528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726268b-f9be-4cd2-a5ad-7dbd0c6becb6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.295681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726268b-f9be-4cd2-a5ad-7dbd0c6becb6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.296246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726268b-f9be-4cd2-a5ad-7dbd0c6becb6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.299267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 439/886] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:01.300797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726268b-f9be-4cd2-a5ad-7dbd0c6becb6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:01.396504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add31f4d-bab7-4d9b-946a-2be0a137d716 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:01.425869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:01.440667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:01.441356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.441637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.441917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.442635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:01.442947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:01.444396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d641455-ecfe-4c7c-adf1-ca00980dd32e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:01.449826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 448/887] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:01.451201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.451488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.451868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.474162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:01.475465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:01.476384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726268b-f9be-4cd2-a5ad-7dbd0c6becb6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:01.527517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:01.528096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.528382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.528862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.565877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 449/888] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:01.588828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:01.639696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:01.651823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:01.654700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:01.771048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7d5d0-cd48-4372-857e-62c1f34ada21 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:01.806824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:01.840353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:01.840353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:01.840353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:01.840353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:01.841858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:01.842687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 440/889] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:01 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:01.843288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:01.844386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:01.860239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:01.861405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:59:01.862267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:01.863053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 450/890] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:59:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:01.870504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:01.986388 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:01.997416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2385748-164c-4a3e-99fd-792a0611f46b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.049376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.050702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.051168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.051602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.067635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.069872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:02.091890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:02.093238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:02.094358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:02.094358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:02.094358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:02.094358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:02.095677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca64ec-7238-409f-a7f5-a69e8a06f74c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.101566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 441/891] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:02.102936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.173205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.174648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.175278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.176102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.177638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:02.178552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.181633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.198104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 451/892] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:59:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:02.201174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.252177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb2b70-8a56-4b3f-9564-7d233bb8e5bb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.297478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:02.299665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.300257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.300589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.300821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.301590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:02.301925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.303360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b23f75-05e8-4176-96c4-25d01e990323 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.320098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 442/893] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:02.321736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94512d6e-f50d-466e-ac23-9f8ee39023a3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.322026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94512d6e-f50d-466e-ac23-9f8ee39023a3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.322303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94512d6e-f50d-466e-ac23-9f8ee39023a3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.324197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 443/894] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:02.330094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94512d6e-f50d-466e-ac23-9f8ee39023a3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.353173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:02.400519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f06aca-6dad-49e3-b427-bd9c46667eb7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.448501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.449148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.449440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.449726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.451420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:02.451420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.452952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b8cb8-4c7a-484d-89c8-a8ab9c8a4f42 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.467416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 452/895] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:59:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:02.467416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.467416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.467416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.487793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:02.489097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:02.551734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94512d6e-f50d-466e-ac23-9f8ee39023a3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.612370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.613499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.613934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.614252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.625371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 453/896] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:02.625371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.628238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.631145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:02.654796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:02.655861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:02.656808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:02.657296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 444/897] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:02.659450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.701703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:02.754574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:02.776844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44014465-cddc-4f45-84d0-b663d20b8729 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.801224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a8b01-72c8-4cdf-99f4-6dfd614eb660 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.823848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.824493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.824775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.825083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.826063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] RBAC: Adding query filter params (interface=public, service_id=59b28619604847cfb3a54371a16bba66, region_id=RegionOne) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:02.826332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.827789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.843978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 454/898] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:59:02 2022] GET /identity/v3/endpoints?service_id=59b28619604847cfb3a54371a16bba66&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:02.843978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.850757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:02.851757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:02.852184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:02.852584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:02.853495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:02.855529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:02.876421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 445/899] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:59:02 2022] POST /identity/v3/services => generated 269 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:02.877812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:02.986238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5cc6-b012-4295-ae4d-bafceb8aa53e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:02.989883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:03.040451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:03.041559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:03.042164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:03.042722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:03.043737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:03.044946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:03.046911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:03.063598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 455/900] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:03.073048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1f2c-b925-4159-9648-5471233fb0f7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:03.073255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:03.124380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:03.145741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd24da-c88e-4031-b144-cd22edea0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:03.146488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd24da-c88e-4031-b144-cd22edea0e32 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:03.146957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd24da-c88e-4031-b144-cd22edea0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:03.147412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd24da-c88e-4031-b144-cd22edea0e32 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:03.148338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd24da-c88e-4031-b144-cd22edea0e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:03.150765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd24da-c88e-4031-b144-cd22edea0e32 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:03.162817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 446/901] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:59:02 2022] GET /identity/v3/services => generated 1268 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:59:03.243320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c2aa6-af2a-4cc6-b1f4-a0924ee581fc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:03.315925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:03.317347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:03.317926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:03.318705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:03.320580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] RBAC: Adding query filter params (role.id=b9ea0ba5c50947e6ad644321a285dc4a, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=564ffb348a9b4554ac2cedb7eced0bb5) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:03.321535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:03.324298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13208375-d15d-43f6-a6aa-5419b2ecf611 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:03.334722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 456/902] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:03 2022] GET /identity/v3/role_assignments?user.id=564ffb348a9b4554ac2cedb7eced0bb5&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=b9ea0ba5c50947e6ad644321a285dc4a => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:04.782100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a64a7c-1a9d-42cb-8176-4a17a0992433 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:04.783622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a64a7c-1a9d-42cb-8176-4a17a0992433 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:04.784408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a64a7c-1a9d-42cb-8176-4a17a0992433 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:04.787734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 447/903] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:04.798420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d993da3-0838-46c4-8628-d10ca7239596 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:04.799928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d993da3-0838-46c4-8628-d10ca7239596 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:04.800706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d993da3-0838-46c4-8628-d10ca7239596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:04.834959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6d993da3-0838-46c4-8628-d10ca7239596 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:04.837672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6d993da3-0838-46c4-8628-d10ca7239596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:04.876499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:04.981203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 457/904] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:05.021267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a64a7c-1a9d-42cb-8176-4a17a0992433 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:05.234783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a64a7c-1a9d-42cb-8176-4a17a0992433 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:05.308385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f743d-8629-471a-bfdb-1e284e26c9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:05.310041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f743d-8629-471a-bfdb-1e284e26c9eb None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:05.310795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f743d-8629-471a-bfdb-1e284e26c9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:05.311556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f743d-8629-471a-bfdb-1e284e26c9eb None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:05.313265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f743d-8629-471a-bfdb-1e284e26c9eb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:05.315755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f743d-8629-471a-bfdb-1e284e26c9eb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:05.333858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287c15f-f2d0-4788-98d2-72b53345a1fc None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:05.335318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287c15f-f2d0-4788-98d2-72b53345a1fc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:05.336657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287c15f-f2d0-4788-98d2-72b53345a1fc None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:05.340320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 458/905] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:05.340689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 448/906] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:59:05 2022] POST /identity/v3/services => generated 241 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:05.346950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:05.347261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:05.347580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:05.375715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:05.377012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:05.486116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:05.542156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 449/907] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:05.565906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287c15f-f2d0-4788-98d2-72b53345a1fc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:05.793279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287c15f-f2d0-4788-98d2-72b53345a1fc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:05.865381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:05.867753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:05.868539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:05.869318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:05.870731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:05.874501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b52a7157-e287-4ebe-ac92-d8ac0497da2e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:05.893561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:05.895474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 16:59:05.897149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:05.902282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 459/908] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:59:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:05.910596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:06.106573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a318e91-a8e5-41e0-b5f8-5f2d6020f2d6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:06.169397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac2b8c-a1f9-41ae-87af-c1076cd3f91c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.169811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac2b8c-a1f9-41ae-87af-c1076cd3f91c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.170300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac2b8c-a1f9-41ae-87af-c1076cd3f91c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.174907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:06.176230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 460/909] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:06.176916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.177671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.178378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.178677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.178842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.179192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.180529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:06.181322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:06.184094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:06.198364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 450/910] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:59:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:06.205005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:06.205206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:06.205288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:06.225385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:06.305112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 461/911] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:06.330732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:06.345557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:06.473009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc640f-1ea9-4486-a7d1-4c99f24ed974 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:06.484833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cceef-aaf4-4639-a52e-4bef5aeb1ecc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:06.530498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:06.531552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.533274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.533774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.535093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:06.535669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:06.537550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:06.546035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 451/912] 10.222.0.48 () {56 vars in 1114 bytes} [Tue Jul 26 16:59:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:06.553016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:06.562535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:06.563485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.563911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.564332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.565167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:06.566739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:06.577989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:06.579182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:59:06.580039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:06.582995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 462/913] 10.222.0.48 () {56 vars in 1132 bytes} [Tue Jul 26 16:59:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:06.593025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:06.610757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:06.757356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ffe39-7e9c-4e33-bec5-005ae51936ec None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:06.763493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66cfafa-9ca7-4305-a817-b92eb647c0a8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:06.778603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:06.827590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:06.828674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.829233 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.829758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.830917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:06.831490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:06.832652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6ffd-672a-449c-a1ed-ca81e4737783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:06.833732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6ffd-672a-449c-a1ed-ca81e4737783 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.834219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6ffd-672a-449c-a1ed-ca81e4737783 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.834686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6ffd-672a-449c-a1ed-ca81e4737783 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.834813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38691cd4-e1f7-40d0-b287-3249458f1abd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:06.836060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6ffd-672a-449c-a1ed-ca81e4737783 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:06.838434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6ffd-672a-449c-a1ed-ca81e4737783 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:06.844868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 463/914] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:06.847231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a7516-0b90-4540-82c2-b89f044faccd None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.847729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a7516-0b90-4540-82c2-b89f044faccd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.848198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a7516-0b90-4540-82c2-b89f044faccd None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.850799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 464/915] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:06.852229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a7516-0b90-4540-82c2-b89f044faccd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:06.886738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 452/916] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:59:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:06.890013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:06.890501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:06.891004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:06.934127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:06.936087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:07.015761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:07.049215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 453/917] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:07.065629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a7516-0b90-4540-82c2-b89f044faccd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:07.071852 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:07.115848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:07.116689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.117114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.117470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.118279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:07.118690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:07.120450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:07.132642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 465/918] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:07.133883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:07.267893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9c714-3d77-4c44-b1e0-41453e125f53 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:07.270365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:07.305919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb28c2-b245-4e24-942d-e69173cd4889 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:07.321939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:07.322615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.322895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.323174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.323956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] RBAC: Adding query filter params (interface=public, service_id=8d49b63d291f4073a5e9576cf6b7ad82, region_id=RegionOne) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:07.324264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:07.325821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:07.332508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 466/919] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:59:07 2022] GET /identity/v3/endpoints?service_id=8d49b63d291f4073a5e9576cf6b7ad82&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:07.338148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:07.344457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:07.396471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:07.398456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.399271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.400338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.425439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:07.437435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:07.438690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:07.439739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:07.443544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 454/920] 10.222.0.48 () {56 vars in 1120 bytes} [Tue Jul 26 16:59:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:07.450670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:07.558451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ad5ff-6151-4e1c-8ebf-36fdfc57b924 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:07.582847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644252a7-465d-40d9-9ff3-6dda0bbaa76d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:07.626122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:07.627338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.627855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.628074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:07.628275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.628920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:07.629264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:07.630730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:07.631392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.631993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.632479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.633782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:07.636284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:07.645366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 455/921] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 16:59:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:07.649186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:07.649840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 467/922] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:59:07 2022] GET /identity/v3/services => generated 1497 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:59:07.667636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:07.784228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f712728-3bbf-49e3-9e2f-15222432c7da None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:07.826030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:07.826701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.826980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.827256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.840590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:07.842725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8117808c-9b44-4cb2-98cd-c582ffdc3512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:07.860920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:07.863548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:07.865788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:07.865788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:07.865788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:07.865788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:07.865788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 456/923] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:07.869348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:07.914678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7eaaf-ddc9-422e-981a-15f542b40713 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.915185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7eaaf-ddc9-422e-981a-15f542b40713 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.915692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7eaaf-ddc9-422e-981a-15f542b40713 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.918784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 457/924] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:07.926819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:07.927324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:07.927814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:07.956654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:07.957991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:08.009970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:08.047628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:08.065233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 458/925] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:08.088454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ffed4-701e-448f-a20d-47cea4f999af None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.090519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.151771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.153171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.154844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.155400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.156913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:08.157543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.160196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.195214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 468/926] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:08.197314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.240850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40ba0e-1372-453c-9d12-25fddca2bb10 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.282156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.283201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.283684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.284191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.285093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.286701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:08.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:08.299040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:08.299887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:08.307803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 459/927] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:08.309155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.383628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:08.441081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee341ba-a0e6-410b-bb8b-440e974edcdb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.447366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421115b-cec5-4321-bb2b-9c8bfb790ed4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.489910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.490904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.491384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.491892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.492798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:08.493354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.494960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.502817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 460/928] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:08.507764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:08.510093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.530674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.532700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.533656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.534434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.553095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.556645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:08.583237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:08.585037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:59:08.586983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:08.591644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 469/929] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:08.597928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.644280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99afb5f-60d0-40d8-9920-d3ebaaa62ca0 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.671221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:08.688487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.689158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.689440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.689710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.690429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:08.690968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.692447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.706351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 461/930] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:08.707377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.833004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edbd22-a2ef-4541-b43d-09be47cda9d1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.854885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0ba73-1a4a-4d0a-99cc-1bd94b53c1a3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:08.874375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.874819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.875098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.875371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.876155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] RBAC: Adding query filter params (interface=public, service_id=6706827092e348babfea4946a4726c0d, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:08.876460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.877908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.889796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 462/931] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/endpoints?service_id=6706827092e348babfea4946a4726c0d&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:08.890727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.937791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:08.938360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:08.938858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:08.939029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:08.939863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:08.940046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:08.941690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:08.979844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 470/932] 10.222.0.48 () {56 vars in 1149 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:08.980863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:08.995945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:09.064663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:09.105213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69008fda-c994-4b37-a424-915a94e42fc3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:09.177315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:09.178424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:09.178915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:09.179409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:09.179987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b9272-4bff-47a0-b6b8-c2bd363c6d1e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:09.180639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:09.183884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:09.201673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 463/933] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/services => generated 1497 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:59:09.271802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:09.272977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:09.273506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:09.274003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:09.301224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:09.304762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:09.342272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:09.343271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:09.344159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:09.344159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:09.344159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:09.344159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:09.345948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 471/934] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:09.351724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:09.508569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea53f6-1dce-44e6-a935-0c879c141167 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:09.525390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:09.566621 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:09.567682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:09.568160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:09.568657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:09.569976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:09.570535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:09.573233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce46f0-3565-432f-9f2e-860587c0b042 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:09.599818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 464/935] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:09.606354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:09.819620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:09.827069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971b881-c197-43cc-aeda-fb535c00de08 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:09.887175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c3a9-9033-4d6e-a8cb-5c966db18e36 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:09.888257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c3a9-9033-4d6e-a8cb-5c966db18e36 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:09.889155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c3a9-9033-4d6e-a8cb-5c966db18e36 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:09.892814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 465/936] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:09.902057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:09.902057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:09.902057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:09.902597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:09.905906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:09.907123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:09.907918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:09.917447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:09.921914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:09.943709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:09.946031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:09.948843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:09.950575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:09.952288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:09.958647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 472/937] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:09.969304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:10.081072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 466/938] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:10.127808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:10.214562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31739b-ea95-42fe-b1c1-c6678dae2e8a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:10.280505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:10.281217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:10.281492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:10.281763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:10.282629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:10.283176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:10.284698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:10.309289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 473/939] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:10.311601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:10.338307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:10.348656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f997af-7e04-4a5e-8ab1-58a787b8ed99 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:10.352313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:10.443774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:10.444576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:10.445031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:10.445571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:10.446366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:10.448002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:10.455833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING keystone.server.flask.common [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 16:59:10.485533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.sql.core [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 16:59:10.485533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 16:59:10.485533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [parameters: {'id': '763487173fe64405b6ea222baa93213d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 16:59:10.485533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80462) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 16:59:10.499794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4992b6-2277-4c59-9b5a-a713740b006d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 763487173fe64405b6ea222baa93213d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 763487173fe64405b6ea222baa93213d. Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:59:10.522722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result.read() Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 16:59:10.524535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application yield resource Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise self.value Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application session.commit() Jul 26 16:59:10.526783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.session.flush() Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application compat.raise_( Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 16:59:10.529341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 16:59:10.531214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application result.read() Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:59:10.532839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [parameters: {'id': '763487173fe64405b6ea222baa93213d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:10.539152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:10.541223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 763487173fe64405b6ea222baa93213d. Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 467/940] 10.222.0.48 () {60 vars in 1140 bytes} [Tue Jul 26 16:59:10 2022] POST /identity/v3/projects => generated 262 bytes in 409 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:10.543348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:10.605753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:10.607508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:10.608281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:10.609068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/564ffb348a9b4554ac2cedb7eced0bb5/roles/b9ea0ba5c50947e6ad644321a285dc4a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:10.619081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=564ffb348a9b4554ac2cedb7eced0bb5, role_id=b9ea0ba5c50947e6ad644321a285dc4a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:10.624137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:10.659073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 474/941] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:59:10 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/564ffb348a9b4554ac2cedb7eced0bb5/roles/b9ea0ba5c50947e6ad644321a285dc4a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:59:10.730973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82826-62a1-4cce-be25-599f04580314 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:10.807785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:10.809427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:10.810695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:10.811415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:10.827254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:10.831068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0722f343-f4db-49b1-bb32-294be15273bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:10.856598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:10.858692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:10.860800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:10.864952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 468/942] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:10.874426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:11.003819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:11.049679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463fcbf-3c4e-4d33-9d2e-377073444a63 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:11.125171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:11.125737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:11.127834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:11.128147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:11.128942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:11.130429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:11.137275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5524bb-35cb-42cd-bfe8-eea5a5ce6c7c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:11.166381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 475/943] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:11.352323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200661f0-6383-4a08-bd10-e0c85c4a684e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:11.353151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200661f0-6383-4a08-bd10-e0c85c4a684e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:11.353518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200661f0-6383-4a08-bd10-e0c85c4a684e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:11.363424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 469/944] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:11 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:11.371738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:11.373157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:11.373157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:11.406766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:11.408783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:11.501218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 476/945] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:11.534202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200661f0-6383-4a08-bd10-e0c85c4a684e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:11.573519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:11.685905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200661f0-6383-4a08-bd10-e0c85c4a684e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:11.726598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:11.727262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:11.727545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:11.727821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:11.728462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:11.730172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:11.740616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:11.741707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:11.742581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 470/946] 10.222.0.48 () {56 vars in 1132 bytes} [Tue Jul 26 16:59:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:11.749150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:11.829185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:11.871816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cb7a4-df71-4d1f-b885-07b96e4a22ef None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:11.913325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:11.914380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:11.914877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:11.915392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:11.916344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:11.916940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:11.918655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52fead-2509-49c6-a2b4-3850186b905d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:11.925063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 477/947] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:11.933949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:11.992708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07701c-81c9-4a62-bc73-b11ab1482a55 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:11.992894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07701c-81c9-4a62-bc73-b11ab1482a55 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:11.993202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07701c-81c9-4a62-bc73-b11ab1482a55 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:11.994586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 478/948] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:12.001576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:12.002014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:12.002398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:12.024088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:12.025349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:12.030735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:12.051446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2267c-4e3c-4928-9c11-31c13e047278 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:12.091485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:12.092432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:12.092756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:12.092930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:12.093591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:12.093899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:12.095318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:12.107499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 471/949] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:12.108723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:12.117916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 479/950] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:12.142664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:12.235298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b136b-a389-42e8-91b3-02b25867f5a2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:12.276453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b222945-4537-4553-b654-6fd32479616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:12.277000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b222945-4537-4553-b654-6fd32479616e None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:12.277294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b222945-4537-4553-b654-6fd32479616e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:12.277550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b222945-4537-4553-b654-6fd32479616e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:12.278973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b222945-4537-4553-b654-6fd32479616e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:12.279909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b222945-4537-4553-b654-6fd32479616e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:12.285362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:12.310720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 472/951] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:59:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:12.345138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1b60b-7241-4516-be79-8e91fd107d4d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:12.407507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:12.408526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:12.408526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:12.408807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:12.412949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:12.412949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:12.422274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:12.424686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:12.426915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 480/952] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:12.442480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b222945-4537-4553-b654-6fd32479616e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:12.447481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:12.609528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b222945-4537-4553-b654-6fd32479616e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:12.649900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:12.650589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:12.650865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:12.651145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:12.651858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:12.652151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:12.653711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:12.660172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 473/953] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:12.670268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:12.791348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424e242-48cb-4e8f-adb2-1c792e569df3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:12.821954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:12.832583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:12.833640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:12.834131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:12.834635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:12.835548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:12.836073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:12.837747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89904302-9e04-4462-8fec-f4f9fd569683 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:12.848615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 481/954] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:12.850368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:12.928879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:12.975738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88905ea-f2e0-47b5-8969-cebb15020154 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:13.017201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f715c88-31c6-42e3-8d31-71069997fcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:13.018110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f715c88-31c6-42e3-8d31-71069997fcae None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.018661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f715c88-31c6-42e3-8d31-71069997fcae None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.019097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f715c88-31c6-42e3-8d31-71069997fcae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.020003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f715c88-31c6-42e3-8d31-71069997fcae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:13.021593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f715c88-31c6-42e3-8d31-71069997fcae None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:13.051420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 474/955] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:59:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:13.156614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a73df3-00f6-48c0-830f-99bc72f0a4de None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.156903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a73df3-00f6-48c0-830f-99bc72f0a4de None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.157213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a73df3-00f6-48c0-830f-99bc72f0a4de None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.159005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 482/956] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:13.164800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.165379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.165854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.191828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:13.193770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:13.287770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 475/957] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:13.313399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a73df3-00f6-48c0-830f-99bc72f0a4de None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:13.502328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a73df3-00f6-48c0-830f-99bc72f0a4de None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:13.568532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372cef81-3867-4fb9-b174-df46aeef3095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:13.569528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cef81-3867-4fb9-b174-df46aeef3095 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.569958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cef81-3867-4fb9-b174-df46aeef3095 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.570425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372cef81-3867-4fb9-b174-df46aeef3095 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.590863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372cef81-3867-4fb9-b174-df46aeef3095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-372cef81-3867-4fb9-b174-df46aeef3095 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:13.601329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:13.602380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:13.603274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:13.604186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 483/958] 10.222.0.48 () {56 vars in 1120 bytes} [Tue Jul 26 16:59:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:13.613356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:13.678042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:13.774293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f341ed-997b-4b76-9347-fefc2139e950 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:13.827467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51226efe-ca70-4336-b720-db4fefb67874 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.827937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51226efe-ca70-4336-b720-db4fefb67874 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.828638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51226efe-ca70-4336-b720-db4fefb67874 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.833009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 484/959] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:13.840178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.842118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.842933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.845559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:13.846211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:13.846491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:13.846775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:13.847528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:13.847837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:13.849321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:13.862057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 476/960] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 16:59:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:13.871553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:13.883318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:13.885722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:13.979413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:14.035714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 485/961] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:14.061885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:14.066590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aec04d-4505-47c4-b546-b0aad6978f38 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:14.083703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:14.116328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:14.117034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.117321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.117590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.133985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:14.135486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:14.156541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:14.158306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:14.160031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:14.160031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:14.160031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:14.160031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:14.160031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 477/962] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:14.167741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:14.306503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5ddca-9bc6-4832-b62e-8e7b04a1949e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:14.376738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b693fac-1aa3-4f6c-a6c6-e1795b29d744 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:14.379065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:14.380748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.382662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.383393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.384825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:14.387669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:14.393267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING keystone.server.flask.common [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 16:59:14.421363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.sql.core [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 16:59:14.421363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 16:59:14.421363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [parameters: {'user_id': '27569eefee2b43c1af44e2d2f9eeb749', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 16:59:14.421363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result.read() Jul 26 16:59:14.439373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:59:14.441387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application yield resource Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application next(self.gen) Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise self.value Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application session.commit() Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 16:59:14.443367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.session.flush() Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application compat.raise_( Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 16:59:14.445255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 16:59:14.447260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application conn.query(q) Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application result.read() Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 16:59:14.449216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [parameters: {'user_id': '27569eefee2b43c1af44e2d2f9eeb749', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:14.453800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 16:59:14.457538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:14.460184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 486/963] 10.222.0.48 () {60 vars in 1131 bytes} [Tue Jul 26 16:59:14 2022] POST /identity/v3/users => generated 160 bytes in 396 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:14.468992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:14.468992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.469599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:14.469810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.469810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.470216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:14.470619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:14.473248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:14.505129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 478/964] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:14.507031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:14.556771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:14.566914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:14.707935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952dfbf-b02c-464c-8b82-a2022bfb70a7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:14.741682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b51c-0bb2-4029-9def-0fcbfaee5651 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:14.791171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:14.792291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.792778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.793557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.809545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:14.813572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:14.821896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:14.822877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.823294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.823679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.838901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8789002e-7844-401a-b2f0-3c725b8eff00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:14.840383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:14.841762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 16:59:14.842756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:14.843327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:14.843403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 487/965] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:59:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:14.845617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3cef-1379-488c-a9bc-6c62aefd1edd None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.845961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3cef-1379-488c-a9bc-6c62aefd1edd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.846241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3cef-1379-488c-a9bc-6c62aefd1edd None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.855768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 488/966] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:14.857687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3cef-1379-488c-a9bc-6c62aefd1edd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-433596ea-8785-434e-bcea-37e583dd4dff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:14.859003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:14.860048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 16:59:14.861083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:14.863531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 479/967] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 16:59:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:14.865153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:14.865616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:14.865898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:14.900491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:14.902673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:14.998297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:15.011056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3cef-1379-488c-a9bc-6c62aefd1edd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:15.021175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 480/968] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:15.022405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:15.058522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:15.079543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:15.080497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.081057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.081541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.082471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:15.082810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:15.084718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:15.148357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 489/969] 10.222.0.48 () {56 vars in 1101 bytes} [Tue Jul 26 16:59:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:15.150141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:15.220828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8c80d-bdc3-4b05-a3b1-4c02dd7985ca None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:15.293080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:15.295420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.296152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.296791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.298516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:15.299263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:15.302456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:15.336953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 481/970] 10.222.0.48 () {56 vars in 1149 bytes} [Tue Jul 26 16:59:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:15.345752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:15.369226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fd26f-edd8-4c82-a1a4-89430f4642dd None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:15.447093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:15.448198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.449419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.451564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.451564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:15.454884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:15.471217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:15.473354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:15.474994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 490/971] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:15.483847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:15.520193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:15.531749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:15.589333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533e829-dce4-4ef4-a65c-2300f460ff78 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:15.657687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16633b-d249-42f5-a409-e2c38af2b3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:15.658606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16633b-d249-42f5-a409-e2c38af2b3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.658986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16633b-d249-42f5-a409-e2c38af2b3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.659354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16633b-d249-42f5-a409-e2c38af2b3e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.667944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe8bce-d351-44a5-a9d3-ad4050640f19 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:15.716089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:15.717116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.717538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.717987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.719154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:15.719650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:15.722029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5032-e586-43ec-bf62-0d4c5c2e6c14 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:15.730317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 491/972] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:15.733148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead273d2-2023-405c-967c-3a89e9fec79b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.733595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead273d2-2023-405c-967c-3a89e9fec79b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.734016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead273d2-2023-405c-967c-3a89e9fec79b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.735989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 492/973] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:15.740684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead273d2-2023-405c-967c-3a89e9fec79b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:15.756074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16633b-d249-42f5-a409-e2c38af2b3e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:15.757175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16633b-d249-42f5-a409-e2c38af2b3e5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:15.842166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:15.854124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:15.876656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead273d2-2023-405c-967c-3a89e9fec79b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:15.885847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 482/974] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:15 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:15.887464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.887790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.888077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.908834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:15.910119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:15.939506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:15.940538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:15.940990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:15.941615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:15.942580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:15.943067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:15.945465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:15.969004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 493/975] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:15.969004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.033920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 483/976] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:16.058932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.136402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99679a-ca7e-45a7-8fa1-13398d916cc0 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:16.188862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:16.189572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:16.189857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:16.190126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:16.204211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:16.205756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:16.219380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:16.226936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:16.228671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:16.230650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:16.230650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:16.230650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:16.230650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:16.230650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 494/977] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:16.239694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.261918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:16.269406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bdc1c-97d9-448f-9839-fa1496fb2b93 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:16.344584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:16.346066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:16.346795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:16.352059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:16.352059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:16.352059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:16.374174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:16.376412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:16.378232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:16.381860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 484/978] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:16.391626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.436184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c24ee-8e8f-40f3-a593-935c91d95ef2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:16.494866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:16.496250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:16.496720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:16.497217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:16.498624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:16.499154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:16.501874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:16.536366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 495/979] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:16.538081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.563087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06c252-d29e-4fe8-a9df-c4c2cbfe0530 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:16.615056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:16.616068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:16.616573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:16.617105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:16.618368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:16.618920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:16.621664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:16.632020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 485/980] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:16.638835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.654929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:16.681294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:16.740092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a7b63-173e-4e55-aa01-27a18d594e92 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:16.805980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:16.806944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:16.807381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:16.807865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:16.818060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:16.820005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:16.822361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28a51c-76bf-4134-a3b0-86f8bdc33dc4 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:16.840009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:16.843095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:16.844836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:16.849192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 496/981] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:16.850168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:16.900017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:16.901150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:16.901747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:16.902225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:16.903547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:16.904111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:16.907002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:16.917329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 486/982] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:16.926886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:17.029230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203d14-0f53-4c78-9da4-939dba4a6ac1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:17.063895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:17.072374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:17.087402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:17.088222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:17.088637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:17.089103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:17.089927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ea2d8-a9ef-42b6-a06e-85ff48f92c8f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:17.090098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:17.090580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:17.092752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:17.115381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 497/983] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:17.117408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:17.149951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:17.150853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:17.151297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:17.151709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:17.152686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] RBAC: Adding query filter params (interface=admin, service_id=6706827092e348babfea4946a4726c0d, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:17.153182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:17.155367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:17.167104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 487/984] 10.222.0.48 () {56 vars in 1249 bytes} [Tue Jul 26 16:59:16 2022] GET /identity/v3/endpoints?service_id=6706827092e348babfea4946a4726c0d&interface=admin®ion_id=RegionOne => generated 193 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:17.168930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:17.301595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683e2e9-fecc-4aa2-90d6-c4f409482a7a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:17.362326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2293a3-8ae3-4550-8251-9ce2974ec827 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:17.366811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:17.367461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:17.367744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:17.368020 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:17.369130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] RBAC: Adding query filter params (role.id=b9ea0ba5c50947e6ad644321a285dc4a, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=564ffb348a9b4554ac2cedb7eced0bb5) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:17.369648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:17.372271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8dce76-2675-4373-bb5c-92ad12b3b30d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:17.382697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 498/985] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:17 2022] GET /identity/v3/role_assignments?user.id=564ffb348a9b4554ac2cedb7eced0bb5&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=b9ea0ba5c50947e6ad644321a285dc4a => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:17.425678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b6a55-3639-4267-a792-bff66e26e26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:17.426251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6a55-3639-4267-a792-bff66e26e26a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:17.426534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6a55-3639-4267-a792-bff66e26e26a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:17.426809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b6a55-3639-4267-a792-bff66e26e26a None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:17.427506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6a55-3639-4267-a792-bff66e26e26a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:17.429259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b6a55-3639-4267-a792-bff66e26e26a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:17.438518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 488/986] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:59:17 2022] GET /identity/v3/services => generated 1497 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 16:59:18.201628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572c96b-340f-4557-8021-81f8294e89c6 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:18.202236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572c96b-340f-4557-8021-81f8294e89c6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:18.202236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572c96b-340f-4557-8021-81f8294e89c6 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:18.203793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 499/987] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:18.209468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254bb4b-a2be-4583-80b9-72765e3a27b1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:18.210122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254bb4b-a2be-4583-80b9-72765e3a27b1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:18.210122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254bb4b-a2be-4583-80b9-72765e3a27b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:18.230891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9254bb4b-a2be-4583-80b9-72765e3a27b1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:18.232688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9254bb4b-a2be-4583-80b9-72765e3a27b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:18.250925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:18.315582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 489/988] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:18.342016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572c96b-340f-4557-8021-81f8294e89c6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:18.397306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:18.480404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572c96b-340f-4557-8021-81f8294e89c6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:18.521127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b8e92-7ca6-430c-acd6-39fa9af3442d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:18.521757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b8e92-7ca6-430c-acd6-39fa9af3442d None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:18.522036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b8e92-7ca6-430c-acd6-39fa9af3442d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:18.522321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b8e92-7ca6-430c-acd6-39fa9af3442d None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:18.523004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b8e92-7ca6-430c-acd6-39fa9af3442d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:18.524472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b8e92-7ca6-430c-acd6-39fa9af3442d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:18.553756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 500/989] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:59:18 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:19.588671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7dad9-9c31-4da4-96f4-9ab818b873a6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:19.589743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7dad9-9c31-4da4-96f4-9ab818b873a6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:19.590239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7dad9-9c31-4da4-96f4-9ab818b873a6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:19.592131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 490/990] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:19.604141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:19.605906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:19.607214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:19.640450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:19.643755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:19.655609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df1709-e38c-44e4-aeb3-9e2d974f511a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:19.655883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df1709-e38c-44e4-aeb3-9e2d974f511a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:19.656337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df1709-e38c-44e4-aeb3-9e2d974f511a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:19.662724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 491/991] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:19.666669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:19.667231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:19.667706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:19.688676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:19.690329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:19.800042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 501/992] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:19.800689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 492/993] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:19.816658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:19.842194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:19.862479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:19.978202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ce511-30b6-4987-a0cf-887db7343531 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.003635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:20.024889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.026246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.026876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.027516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] PATH_INFO: `/v3/services/image` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.028945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.032111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.038839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142cb50-9399-4a5f-bcc2-5882bcb11d9b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:20.056847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:20.057982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:20.058854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:20.060419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 502/994] 10.222.0.48 () {56 vars in 1105 bytes} [Tue Jul 26 16:59:19 2022] GET /identity/v3/services/image => generated 86 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:20.069667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:20.095427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.096721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.097044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.097321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.098160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.099850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:20.115360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:20.117393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 16:59:20.119074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:20.123591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 493/995] 10.222.0.48 () {56 vars in 1126 bytes} [Tue Jul 26 16:59:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:20.124912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:20.271210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:20.278695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebef1c-c0b5-4406-b35e-f1976833e958 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.285397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c269-bfb6-4901-9979-42c096f5391a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.347175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.348268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.348765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.349283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.350573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:20.351135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.357101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.361289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.362343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.362780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.363259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.364524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] RBAC: Adding query filter params (name=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:20.365103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.370575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 494/996] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:20.370775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.373496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:20.382935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 503/997] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:20.383310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:20.385024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:20.567214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41e772-2371-4aa7-9d18-1ff131b0a031 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.574605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22effe9-1ab4-421b-ae81-f773a268bd7d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.612626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.613912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.614426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.614732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.615420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:20.615726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.617230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.624052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 495/998] 10.222.0.48 () {56 vars in 1122 bytes} [Tue Jul 26 16:59:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:20.630207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:20.646660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.647758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.648949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.649476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.651629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] RBAC: Adding query filter params (type=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:20.652171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.656769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed4079-4f10-42a7-ad6d-25153421563d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.673488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 504/999] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:20.723399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:20.772077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39f060-6739-4ce3-9d9d-85b853bb1957 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:20.816031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9af843-674c-44e2-9344-a614bb49a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:20.816670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9af843-674c-44e2-9344-a614bb49a288 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:20.817011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9af843-674c-44e2-9344-a614bb49a288 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:20.817348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9af843-674c-44e2-9344-a614bb49a288 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:20.818023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9af843-674c-44e2-9344-a614bb49a288 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:20.819470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9af843-674c-44e2-9344-a614bb49a288 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:20.839431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 496/1000] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:59:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:21.073870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce294c78-8d71-45b5-8db4-2f6278f17c8d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:21.074113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce294c78-8d71-45b5-8db4-2f6278f17c8d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:21.074441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce294c78-8d71-45b5-8db4-2f6278f17c8d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:21.076256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 505/1001] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:21.083700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:21.084333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:21.084936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:21.118337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:21.120456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:21.264546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 497/1002] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:21 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:21.308232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce294c78-8d71-45b5-8db4-2f6278f17c8d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:21.374827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:21.542216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce294c78-8d71-45b5-8db4-2f6278f17c8d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:21.614181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:21.615336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:21.615853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:21.616401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:21.617630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:21.620256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:21.642318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:21.644228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:21.657585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 506/1003] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:21.660813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:21.773569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:21.868843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cdc70-8257-48e5-90fa-932769da1622 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:21.936110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:21.937027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:21.937540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:21.938018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:21.939477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:21.940032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:21.942997 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:21.953868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 498/1004] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:21.962939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:22.151917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:22.184852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5663f8-3eb5-4a93-8998-1ac3f2f5c8a7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:22.255907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:22.256897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:22.257444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:22.257887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:22.259107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:22.259647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:22.262285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:22.273682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 507/1005] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:22.283398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:22.506092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8bb1-0bf0-4e00-8419-4dd432b2a2ad None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:22.544712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:22.582228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:22.583281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:22.583772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:22.584262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:22.585783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] RBAC: Adding query filter params (interface=public, service_id=fcf46f7fa1c4414f9d131b38e9e9a802, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:22.586361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:22.589054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe9013-7962-4b9c-91a4-1ce8fc76d7a6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:22.600039 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 499/1006] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:59:22 2022] GET /identity/v3/endpoints?service_id=fcf46f7fa1c4414f9d131b38e9e9a802&interface=public®ion_id=RegionOne => generated 194 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:22.610196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:22.824894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfef8b2-ec16-4e79-9e50-68b89e2bd4c2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:22.895148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362d1d6-995c-4aca-b1d1-32a3a9c864a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:22.896210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362d1d6-995c-4aca-b1d1-32a3a9c864a8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:22.896670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362d1d6-995c-4aca-b1d1-32a3a9c864a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:22.897161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362d1d6-995c-4aca-b1d1-32a3a9c864a8 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:22.898365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362d1d6-995c-4aca-b1d1-32a3a9c864a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:22.901016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362d1d6-995c-4aca-b1d1-32a3a9c864a8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:22.921350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 508/1007] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:59:22 2022] GET /identity/v3/services => generated 1727 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:59:23.004069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00fe13-9212-4b61-9c58-6b4c25c98c91 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.004609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00fe13-9212-4b61-9c58-6b4c25c98c91 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.005121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00fe13-9212-4b61-9c58-6b4c25c98c91 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.007777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 500/1008] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:23.015252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.015752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.016221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.044143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:23.046591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:23.059798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:23.071846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965c8c0-bcc4-4631-bc4d-5883e2a02f25 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.072260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965c8c0-bcc4-4631-bc4d-5883e2a02f25 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.072647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965c8c0-bcc4-4631-bc4d-5883e2a02f25 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.074322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 501/1009] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:23.083605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.084069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.084523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.120678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:23.123676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:23.184625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 509/1010] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:23.215885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:23.247930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 502/1011] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:23.271356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:23.414591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe551d74-d6bd-48ec-b721-ccf45e46b5bb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:23.430216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2251d11-1efc-46a6-b5bc-61a1ff98ad8c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:23.443093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:23.473323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:23.474122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.474506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.475043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.475715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:23.477273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:23.501211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80da27-cc5c-4bc4-854f-74781fa6a515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:23.502348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80da27-cc5c-4bc4-854f-74781fa6a515 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.502855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80da27-cc5c-4bc4-854f-74781fa6a515 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.503351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80da27-cc5c-4bc4-854f-74781fa6a515 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.505648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80da27-cc5c-4bc4-854f-74781fa6a515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:23.506251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 503/1012] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:59:23 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:23.508320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80da27-cc5c-4bc4-854f-74781fa6a515 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:23.512052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 510/1013] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:23.518833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:23.741169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ca112d-ffb4-42f6-9acc-10b087aae82d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:23.782564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7d29a-f8bd-4683-9a10-146e86f9f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:23.783324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7d29a-f8bd-4683-9a10-146e86f9f48c None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:23.783674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7d29a-f8bd-4683-9a10-146e86f9f48c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:23.784020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7d29a-f8bd-4683-9a10-146e86f9f48c None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:23.784811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7d29a-f8bd-4683-9a10-146e86f9f48c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:23.786418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa7d29a-f8bd-4683-9a10-146e86f9f48c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:23.820529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 504/1014] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:59:23 2022] POST /identity/v3/projects => generated 311 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:25.238523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc9956-c37d-4d1a-927c-7095c9126158 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:25.239346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc9956-c37d-4d1a-927c-7095c9126158 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:25.239346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc9956-c37d-4d1a-927c-7095c9126158 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:25.240891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 511/1015] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:25.246709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:25.248015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:25.248015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:25.269752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:25.271049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:25.291730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:25.376039 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 505/1016] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:25 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:25.405350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc9956-c37d-4d1a-927c-7095c9126158 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:25.429161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:25.550778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc9956-c37d-4d1a-927c-7095c9126158 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:25.593847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:25.594522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:25.594808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:25.595087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:25.595737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:25.597207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:25.608784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:25.610161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:25.611819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 512/1017] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:25.619569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:25.746424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11ba1c-5a79-49f5-ae43-a4bb58e1270b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:25.792085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:25.792908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:25.793358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:25.793768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:25.794668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:25.795114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:25.796908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d27325-ef46-452b-b87a-48ce1a35ef31 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:25.808154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 506/1018] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:25.809460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:25.927362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:25.975819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaa4cc-3abc-49d4-8c70-738f99d4685c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:25.999625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a5da4-8a7c-4f14-835d-6e7c1c2f07c3 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:25.999926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a5da4-8a7c-4f14-835d-6e7c1c2f07c3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.000239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a5da4-8a7c-4f14-835d-6e7c1c2f07c3 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.001890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 507/1019] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:26.007617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.007965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.008211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.029540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:26.030905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:26.042947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:26.045767 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:26.046419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.046695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.047260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.047693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:26.048169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:26.049493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:26.055547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 513/1020] 10.222.0.48 () {56 vars in 1112 bytes} [Tue Jul 26 16:59:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:26.061742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:26.154764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 508/1021] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:26 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:26.197236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:26.244306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91253ac7-edc3-427d-8f27-d7616201940c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:26.310740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3356e8-cb5c-4a39-891c-d30f6cfcca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:26.311614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3356e8-cb5c-4a39-891c-d30f6cfcca83 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.312141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3356e8-cb5c-4a39-891c-d30f6cfcca83 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.312524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3356e8-cb5c-4a39-891c-d30f6cfcca83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.313622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3356e8-cb5c-4a39-891c-d30f6cfcca83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:26.315786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3356e8-cb5c-4a39-891c-d30f6cfcca83 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:26.332175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:26.356294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 514/1022] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:59:26 2022] POST /identity/v3/endpoints => generated 334 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:26.382392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec489d-35e1-4da2-b252-68132103a4a6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:26.410588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08221-d9df-4268-9b9e-6fe9e0e43d36 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.411128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08221-d9df-4268-9b9e-6fe9e0e43d36 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.411591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08221-d9df-4268-9b9e-6fe9e0e43d36 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.415278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 515/1023] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:26.424618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.425159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.425642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.455025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:26.456633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:26.474000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:26.475098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.475231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.475542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.477113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:26.479549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:26.483694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 509/1024] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:26.496473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:26.513914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:26.585058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 516/1025] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:26.608761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:26.683145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e835bb-3c03-4534-8621-de5da2ac3192 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:26.743131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:26.743777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.744696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.745091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.745829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:26.748146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:26.777784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 510/1026] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:59:26 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 16:59:26.798114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a4b09-6983-4daa-92df-9b3f0098c52a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:26.836563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:26.851266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:26.852170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:26.852375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:26.852629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] PATH_INFO: `/v3/services/image` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:26.853264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:26.854899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:26.866198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:26.873525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:26.874608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 517/1027] 10.222.0.48 () {56 vars in 1105 bytes} [Tue Jul 26 16:59:26 2022] GET /identity/v3/services/image => generated 86 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:26.880303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:27.014206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:27.093545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e230fcb-14e2-4393-bf45-92a9e89b400f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:27.159789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:27.160830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:27.161316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:27.161765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:27.162892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:27.163347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:27.165523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:27.177062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 511/1028] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:27.181458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:27.387133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d026f-ea2a-469a-8c98-6da433fd3a79 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:27.455295 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:27.456353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:27.456844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:27.457370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:27.458689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] RBAC: Adding query filter params (type=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:27.459220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:27.461898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:27.473447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 518/1029] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:27.480360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:27.604518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ca682-9159-4bc9-aab3-8d1afba6cce3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:27.635913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:27.646444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:27.646980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:27.647272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:27.647528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:27.648329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] RBAC: Adding query filter params (interface=public, service_id=f456f79974d4485084b1e1cc57ba7b25, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:27.648639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:27.650169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ae185-f63b-49c5-9f8f-f29118000723 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:27.657555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 512/1030] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 16:59:27 2022] GET /identity/v3/endpoints?service_id=f456f79974d4485084b1e1cc57ba7b25&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:27.663157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:27.757018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:27.818453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d58077-f3e1-4c7f-8d67-2852da2d6058 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:27.860437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfc6db-fce5-4f09-ba8d-22407f481c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:27.861112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfc6db-fce5-4f09-ba8d-22407f481c8a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:27.861399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfc6db-fce5-4f09-ba8d-22407f481c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:27.861671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfc6db-fce5-4f09-ba8d-22407f481c8a None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:27.862332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfc6db-fce5-4f09-ba8d-22407f481c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:27.863741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfc6db-fce5-4f09-ba8d-22407f481c8a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:27.873617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 519/1031] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 16:59:27 2022] GET /identity/v3/services => generated 1957 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 16:59:28.938590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238589b4-9fe6-4a16-bc25-e1ed3cdd902a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:28.939162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238589b4-9fe6-4a16-bc25-e1ed3cdd902a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:28.939585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238589b4-9fe6-4a16-bc25-e1ed3cdd902a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:28.945028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 513/1032] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:28.948831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:28.949600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:28.949600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:28.979744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:28.982472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:29.081570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8b18-4f63-463f-9e03-c6984fadc8ea None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.082011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8b18-4f63-463f-9e03-c6984fadc8ea None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.082468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b8b18-4f63-463f-9e03-c6984fadc8ea None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.084230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 514/1033] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:29.091081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.091521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.091861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.118063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:29.120442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:29.125966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 520/1034] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:29.149760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:29.246218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:29.247985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 515/1035] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:29.273007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:29.333734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb7cd6-8241-4987-8d7e-2063d5055d82 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:29.397717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:29.398423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.398704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.398971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.413770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:29.418624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:29.421422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:29.423156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:29.425842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 521/1036] 10.222.0.48 () {56 vars in 1120 bytes} [Tue Jul 26 16:59:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:29.426993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:29.461336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aef2deb-6d7c-475f-a656-2f4605fb56f8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:29.507695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:29.529829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:29.530954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.531443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.531930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.552808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:29.563135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:29.564164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:59:29.565083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:29.566601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 516/1037] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:59:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:29.575472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:29.600347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec646044-e28f-4b57-b6f4-7ed3034320f1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:29.621442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:29.667261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:29.668249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.668698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.670271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.671306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:29.671769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:29.673405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:29.690231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 522/1038] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 16:59:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:29.692820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:29.746982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893fec7-9857-4d96-8fc3-488e719124fd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:29.795256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:29.795977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.796274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.796547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.797328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:29.797630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:29.799061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:29.811572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 517/1039] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:59:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:29.818625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:29.885481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e2f48-e895-4857-a814-fa9064a85923 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:29.906702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:29.950090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:29.951686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:29.952149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:29.952603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:29.971434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:29.973208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-7bbbab80-b65f-43bd-a5d2-e88d7d12bd95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:29.992166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:29.994050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:29.995101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:29.995101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:29.995101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:29.995101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:29.995101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 523/1040] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:30.001041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff8734-06fb-4673-b7e1-9a40a309c980 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.001440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff8734-06fb-4673-b7e1-9a40a309c980 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.001629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff8734-06fb-4673-b7e1-9a40a309c980 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.003264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 524/1041] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:30.004044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff8734-06fb-4673-b7e1-9a40a309c980 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.015630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe72e68-6def-4870-a029-f54ee32954a6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.040864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:30.058293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dbb97-94b2-4bdc-9472-8c93263ab28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.059396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dbb97-94b2-4bdc-9472-8c93263ab28f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.060460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dbb97-94b2-4bdc-9472-8c93263ab28f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.061010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dbb97-94b2-4bdc-9472-8c93263ab28f None admin] PATH_INFO: `/v3/users/23d374c27a0148949b35a45981d6d694` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.079092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dbb97-94b2-4bdc-9472-8c93263ab28f None admin] RBAC: Authorizing `identity:get_user(user_id=23d374c27a0148949b35a45981d6d694)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.080554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dbb97-94b2-4bdc-9472-8c93263ab28f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.082375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 518/1042] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:29 2022] GET /identity/v3/users/23d374c27a0148949b35a45981d6d694 => generated 285 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:30.083605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.083888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.084163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.102660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:30.103902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:30.152294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff8734-06fb-4673-b7e1-9a40a309c980 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.191864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.192521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.192823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.193201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.194081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:30.194386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.195829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.210060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 525/1043] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:30.211675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.225604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 519/1044] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:30.226849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.281669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:30.347456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:30.366920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12698813-eec2-4b6a-b1f3-6f4cc4f1a9f7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.380234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19c58-9ed4-45a5-8993-dd949a40db3a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.417474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.418509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.418993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.419544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.422200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.422852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.423153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.423410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.433980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.437431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:30.461110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:30.462310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:30.463312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 520/1045] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:30.464611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.511438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.512999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.615973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:30.616372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:30.633353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a992d-57ac-4f7c-95e6-2efd7147f892 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.639747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 526/1046] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:30.640669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.685424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.686200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.686554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.686901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.687635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.689183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:30.701773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:30.703871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 16:59:30.705506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:30.709220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 521/1047] 10.222.0.48 () {56 vars in 1105 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/services/image => generated 86 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:30.711048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.789624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6380e1b-72ab-4f8b-a10f-33e3d2c4f6e8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.848267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.849441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.849943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.850558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.852041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:30.852711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.855784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.880373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617dd46b-d989-4e81-be80-79c6ad1f67a7 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:30.886156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 527/1048] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:30.887398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.929221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:30.929910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:30.930230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:30.930469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:30.931715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] RBAC: Authorizing `identity:get_project(project_id=ee7c2557de664bfca510590a9110d277)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:30.933217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:30.935628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 522/1049] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/projects/ee7c2557de664bfca510590a9110d277 => generated 311 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:30.936515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:30.944859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:30.970135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:31.035055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcddb4-04a0-4ad4-ab4b-d79c55385824 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.079614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8856c-cca8-48f7-9710-1d0cc9a04d64 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.080660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.081498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.081866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.082279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.083072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] RBAC: Adding query filter params (name=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:31.083480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:31.085026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:31.093045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 528/1050] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:31.093593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:31.122307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.122957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.123257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.123594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.210472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:31.211889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:31.242645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:31.244999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4808d51-aad5-4350-918e-834f3f831d38 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.249249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:31.296618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.297441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.297840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.298199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.299308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=ee7c2557de664bfca510590a9110d277, user.id=23d374c27a0148949b35a45981d6d694) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:31.299626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:31.301349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:31.308951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 529/1051] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:31 2022] GET /identity/v3/role_assignments?user.id=23d374c27a0148949b35a45981d6d694&scope.project.id=ee7c2557de664bfca510590a9110d277&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:31.311978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:31.374460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 523/1052] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:31.385497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:31.476319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d32bdf-f006-43a6-83ad-9d3dce94bc4d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.533601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.534644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.534940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.535309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.536094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:31.536397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:31.537842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:31.551896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 530/1053] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:31.553200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:31.572456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e637dc-d2b7-4941-b29d-56dc8c5b3e3c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.609297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:31.615961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:31.627638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.628481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.628886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.629347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.651883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:31.654812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:31.683267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:31.684262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:31.686000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:31.686000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:31.686000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:31.686000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:31.687148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 524/1054] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:31.695361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:31.742376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfbf1d-5b7f-4832-bf2c-4f84bf9aa0e7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.797637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.798280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.798558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.798832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.799572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:31.801008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:31.835631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 531/1055] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 16:59:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:31.900424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0930c2-b46a-4aa6-8f1e-9f72cd6d77fb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:31.995552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:31.996517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:31.997053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:31.998200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:31.999442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:31.999979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:32.002772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:32.028555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 525/1056] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:32.034500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:32.213630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:32.247998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107c4af-f223-4e03-9700-7d3adf750c9c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:32.294764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:32.295437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:32.295715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:32.296197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:32.304866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:32.306393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:32.320382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:32.330019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:32.331875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 532/1057] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:32.337426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:32.397226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:32.537476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad071-00e2-43a9-9f6e-a905ac260b9b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:32.598449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:32.599609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:32.599982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:32.600475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:32.601994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:32.602515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:32.605502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c298fb3-b392-4199-9ff1-71086c5ff7dc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:32.627416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 526/1058] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:32.636851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:32.853369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfefe2-f53e-4b23-bc5b-c63f32048852 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:32.924167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:32.925348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:32.925848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:32.926341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:32.927862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] RBAC: Adding query filter params (role.id=b9ea0ba5c50947e6ad644321a285dc4a, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=a903f3771a62447d9c06fbeba414e9e3) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:32.928435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:32.931349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815643d-2bb1-425f-bbf1-bf4a6fa8e399 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:32.949004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 533/1059] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:32 2022] GET /identity/v3/role_assignments?user.id=a903f3771a62447d9c06fbeba414e9e3&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=b9ea0ba5c50947e6ad644321a285dc4a => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:33.603969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1a880-6096-43a6-afbd-57dc96b818c6 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:33.604548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1a880-6096-43a6-afbd-57dc96b818c6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:33.605079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1a880-6096-43a6-afbd-57dc96b818c6 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:33.608581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 527/1060] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:33.615322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:33.616102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:33.616102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:33.638225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:33.639159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:33.649867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:33.733347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 534/1061] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:33.756020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1a880-6096-43a6-afbd-57dc96b818c6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:33.881080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:33.895133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1a880-6096-43a6-afbd-57dc96b818c6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:33.938400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:33.939076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:33.939359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:33.939638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:33.953644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:33.958725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:33.960521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:59:33.962133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:33.965502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 528/1062] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:33.966886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:34.094542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615e5f-9a33-422c-bdcc-6d7693162d66 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:34.143480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:34.144127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:34.144402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:34.144681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:34.145459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:34.145771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:34.147192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:34.158341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 535/1063] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:34.164899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:34.358754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6f206-c9e3-430b-a934-08bb6632b3f6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:34.404345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:34.405054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:34.405348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:34.405632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] PATH_INFO: `/v3/users/23d374c27a0148949b35a45981d6d694` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:34.406943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] RBAC: Authorizing `identity:get_user(user_id=23d374c27a0148949b35a45981d6d694)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:34.408437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:34.410483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 529/1064] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:34 2022] GET /identity/v3/users/23d374c27a0148949b35a45981d6d694 => generated 285 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:34.416877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:34.424747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:34.618479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc369cb-195a-45ea-aa51-9b4a417dfa3e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:34.682559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603937d-4c5f-448d-a2ce-a58b622afeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:34.683990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603937d-4c5f-448d-a2ce-a58b622afeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:34.684564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603937d-4c5f-448d-a2ce-a58b622afeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:34.685616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603937d-4c5f-448d-a2ce-a58b622afeb6 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:34.687806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603937d-4c5f-448d-a2ce-a58b622afeb6 None admin] RBAC: Authorizing `identity:get_project(project_id=ee7c2557de664bfca510590a9110d277)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:34.690589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603937d-4c5f-448d-a2ce-a58b622afeb6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:34.698961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 536/1065] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:59:34 2022] GET /identity/v3/projects/ee7c2557de664bfca510590a9110d277 => generated 311 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:34.701480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:34.747137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:34.902321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57062a-181b-4279-afb7-f4605f53ce42 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:34.962871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5513a0-2d94-4315-9af6-e81c516ff783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:34.963999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5513a0-2d94-4315-9af6-e81c516ff783 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:34.964476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5513a0-2d94-4315-9af6-e81c516ff783 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:34.964993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5513a0-2d94-4315-9af6-e81c516ff783 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277/users/23d374c27a0148949b35a45981d6d694/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:34.968944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5513a0-2d94-4315-9af6-e81c516ff783 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee7c2557de664bfca510590a9110d277, user_id=23d374c27a0148949b35a45981d6d694, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:34.971613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5513a0-2d94-4315-9af6-e81c516ff783 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:34.995087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 530/1066] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:59:34 2022] PUT /identity/v3/projects/ee7c2557de664bfca510590a9110d277/users/23d374c27a0148949b35a45981d6d694/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:59:35.242836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc9af0-d036-409e-b694-4994d9f402e8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.243078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc9af0-d036-409e-b694-4994d9f402e8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.243474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc9af0-d036-409e-b694-4994d9f402e8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.244922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 537/1067] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:35.251888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205816-5fc6-4dd2-ad9d-eb830f5bb7de None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.252506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205816-5fc6-4dd2-ad9d-eb830f5bb7de None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.253038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63205816-5fc6-4dd2-ad9d-eb830f5bb7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.287095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-63205816-5fc6-4dd2-ad9d-eb830f5bb7de None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:35.289171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-63205816-5fc6-4dd2-ad9d-eb830f5bb7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:35.429318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 531/1068] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:35.437336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c13db0-1062-47fb-96bf-53c1e57ac9a2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.437624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c13db0-1062-47fb-96bf-53c1e57ac9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.437954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c13db0-1062-47fb-96bf-53c1e57ac9a2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.439455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 538/1069] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:35.445732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.446133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.446567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.473567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c13db0-1062-47fb-96bf-53c1e57ac9a2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:35.476401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:35.478418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:35.485203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:35.589016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 532/1070] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:35.604766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:35.611472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:35.659902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c13db0-1062-47fb-96bf-53c1e57ac9a2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:35.721820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:35.723443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.723964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.724431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.740591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:35.746673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:35.749862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:35.751467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 539/1071] 10.222.0.48 () {56 vars in 1120 bytes} [Tue Jul 26 16:59:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:35.755481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:35.816913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a6cfe-72b8-4cc4-add3-7d3e13e6181c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:35.883826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:35.885303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.886040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.886525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.913356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:35.927684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763ee63-fa00-4f1c-923e-7b6c4e96950d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:35.967810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:35.981018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:35.982105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:35.982671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:35.983113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:35.984300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:35.984785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:35.987095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.009341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 540/1072] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 16:59:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:36.010460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:36.030272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:36.033499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.171831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4f337-4650-49cc-8186-d39841b2d18a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:36.220765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 533/1073] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:36.228598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:36.235960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:36.237101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:36.237598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:36.238098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:36.259814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:36.262629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.289293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:36.295161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:36.296186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:36.297112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:36.297112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:36.297112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:36.297112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:36.297779 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 541/1074] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:36.306791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:36.393031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:36.407724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931bab1-ba8e-4178-aa80-32be50870336 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:36.471952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:36.472970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:36.473486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:36.473980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:36.493539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:36.496364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.505381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb668218-3d1e-4d23-8c49-a17f1e3bdb94 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:36.527535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:36.528756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:36.529770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:36.529770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:36.529770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:36.529770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:36.529770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 534/1075] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:36.540014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:36.573230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:36.573866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:36.574178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:36.574406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:36.575213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:36.575677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:36.577416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.593349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 542/1076] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:36.600089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:36.705432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:36.721676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5e2ff-ece4-4aaa-9b98-fa4def2935d6 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:36.774942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:36.775473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:36.775759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:36.776060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:36.776899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:36.777251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:36.778746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.784824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84d131-bc7f-40e2-8dc0-a7b0577b2e5a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:36.795497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 535/1077] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:36.808203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:36.870196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:36.871345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:36.871842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:36.873096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:36.889458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:36.892429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:36.917924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:36.919686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:36.926394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 543/1078] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:59:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:36.932361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.122290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84239563-a77c-4c38-9bed-f3a5c581b153 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:37.179772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:37.180424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.180740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.181125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.181970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:37.182280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:37.183778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:37.207545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 536/1079] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:59:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:37.217300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.336793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58679ea-35c2-4257-94c8-a1ce34ae1559 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:37.384059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:37.384698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.385090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.385438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.398949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:37.400440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-63fb544b-104f-401d-811f-c65119a1598c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:37.417771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:37.420403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:37.422754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:37.422754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:37.422754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:37.422754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:37.422754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 544/1080] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:37.427027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.452478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:37.586072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4ee7f-5e03-4bed-9214-2ae670d9ff52 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.586522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4ee7f-5e03-4bed-9214-2ae670d9ff52 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.586967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4ee7f-5e03-4bed-9214-2ae670d9ff52 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.588951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 545/1081] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:37.595439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.595827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.596291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.625693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9843c-548c-4e82-80cb-9fde91bad319 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:37.631987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:37.634027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:37.652277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:37.667226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:37.667875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.668155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.668431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.669887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:37.670372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:37.672203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:37.689615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 537/1082] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:37.697183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.724801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 546/1083] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:37.752327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.799468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:37.841600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579d41-5946-4d08-bac3-5d6679a70716 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:37.884220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:37.885364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.885829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.886201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.900035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:37.901472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be90081-c05f-4b6b-bc62-c04e6644fe02 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:37.902797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:37.923358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:37.925335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:37.927086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 538/1084] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:37.931768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.947381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:37.948048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:37.948326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:37.948607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:37.961940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:37.967094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:37.969251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:37.971256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 547/1085] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 16:59:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:37.977555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:37.991082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:38.074138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cda39-b603-41dd-b572-79363b415288 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:38.110993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:38.121525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:38.123993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:38.124483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:38.124935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:38.126077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:38.126584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:38.129145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:38.130249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42254b-35a3-485e-8e73-9e0b6c8d820d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:38.157596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 539/1086] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:38.159099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:38.179378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:38.180011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:38.180302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:38.180561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:38.181349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:38.181651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:38.183097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:38.193483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 548/1087] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 16:59:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:38.199968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:38.313884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:38.319611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36431e2-65fc-42e9-8d2a-67e236b0cd25 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:38.355660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c6a17-e4d7-4f50-b56f-381cd2c02914 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:38.371151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:38.371940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:38.373856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:38.373856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/a903f3771a62447d9c06fbeba414e9e3/roles/b9ea0ba5c50947e6ad644321a285dc4a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:38.376373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=a903f3771a62447d9c06fbeba414e9e3, role_id=b9ea0ba5c50947e6ad644321a285dc4a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:38.378019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:38.395265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 540/1088] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:59:38 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/a903f3771a62447d9c06fbeba414e9e3/roles/b9ea0ba5c50947e6ad644321a285dc4a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 16:59:38.410923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:38.411508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:38.411789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:38.412062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] PATH_INFO: `/v3/users/23d374c27a0148949b35a45981d6d694` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:38.413418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] RBAC: Authorizing `identity:get_user(user_id=23d374c27a0148949b35a45981d6d694)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:38.414936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:38.416753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 549/1089] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:38 2022] GET /identity/v3/users/23d374c27a0148949b35a45981d6d694 => generated 285 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:38.423987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:38.499910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:38.572647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e516f0-59d1-4516-ae41-e65350fd3d63 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:38.631343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:38.631982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:38.632251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:38.632523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:38.715616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:38.717581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:38.777343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:38.845520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 541/1090] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:38.852714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:38.984533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c783539-1d53-4390-b4aa-c4ed6e3fcf86 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:39.025082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556dbc68-f184-4101-8cfa-3c9b430a9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:39.025740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dbc68-f184-4101-8cfa-3c9b430a9ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:39.026014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dbc68-f184-4101-8cfa-3c9b430a9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:39.026278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556dbc68-f184-4101-8cfa-3c9b430a9ef5 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:39.027516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dbc68-f184-4101-8cfa-3c9b430a9ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=ee7c2557de664bfca510590a9110d277)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:39.029142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556dbc68-f184-4101-8cfa-3c9b430a9ef5 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:39.030924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 550/1091] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:59:38 2022] GET /identity/v3/projects/ee7c2557de664bfca510590a9110d277 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:39.038138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:39.225145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c0ce2-e750-44aa-a78a-37dc27a6d28c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:39.267204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:39.267735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:39.268014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:39.268290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:39.269144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=ee7c2557de664bfca510590a9110d277, user.id=23d374c27a0148949b35a45981d6d694) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:39.269453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:39.270938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034cf9e3-d1df-4da5-b9d2-5f625c659870 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:39.278012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 542/1092] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:39 2022] GET /identity/v3/role_assignments?user.id=23d374c27a0148949b35a45981d6d694&scope.project.id=ee7c2557de664bfca510590a9110d277&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:40.534626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede54ca-5b11-443f-949f-b2a2487f5f79 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:40.535424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede54ca-5b11-443f-949f-b2a2487f5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:40.535424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede54ca-5b11-443f-949f-b2a2487f5f79 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:40.536550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 551/1093] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:40.545250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:40.546245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:40.546245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:40.567620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:40.569005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:40.581436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:40.657885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 543/1094] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:40.683065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede54ca-5b11-443f-949f-b2a2487f5f79 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:40.695588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:40.807155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede54ca-5b11-443f-949f-b2a2487f5f79 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:40.849440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:40.850110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:40.850385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:40.850654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:40.864533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 16:59:40.869662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:40.871570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:40.874533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 552/1095] 10.222.0.48 () {56 vars in 1120 bytes} [Tue Jul 26 16:59:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 16:59:40.878837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.085616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d52e2-e440-4b5f-ab21-b8caff94df81 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:41.140171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:41.140951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.141369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.141760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.142636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:41.143066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:41.144588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:41.154853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 544/1096] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 16:59:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:41.160949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.242553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:41.310869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad9629-9314-4eb4-9ab5-ffaaa3a9c291 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:41.371140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:41.371712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.371994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.373151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.387529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:41.389523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6d216f9e-7f38-4681-8967-1cf46b01f96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:41.410763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:41.412827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:41.414857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:41.414857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:41.414857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:41.414857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 16:59:41.414857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 553/1097] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:41.422420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.439939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:41.444666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7941-5373-47b8-8c5e-637f0c1efe70 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.444954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7941-5373-47b8-8c5e-637f0c1efe70 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.445269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec7941-5373-47b8-8c5e-637f0c1efe70 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.446755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 554/1098] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:41.452685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.452993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.453279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.473884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:41.475153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:41.556328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 555/1099] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:41.582623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.607433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fe4b9-a7cd-4b0a-b1c1-d3f46dae172b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:41.649402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:41.649656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:41.650923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.651413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.651898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.653326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:41.655001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:41.657715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:41.675856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 545/1100] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:41.681983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.726253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6baae5-8afb-4345-8d18-949c25437a91 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:41.764771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:41.773905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:41.774583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.774871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.775145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.776395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:41.777914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:41.779977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 556/1101] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:41.786003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.827466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbd4f-7d2b-406e-aaad-a3790bdc7b9c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:41.869016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:41.869681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:41.869965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:41.870242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:41.879250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:41.880640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 16:59:41.894693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:41.895749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:41.897633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 546/1102] 10.222.0.48 () {56 vars in 1093 bytes} [Tue Jul 26 16:59:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:41.904500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:41.944579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d00119-4b91-4523-b750-fb5f202bfc56 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:41.980049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:42.017603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:42.018699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:42.019149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:42.019637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:42.020914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:42.023774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:42.032347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d7848-5d76-43eb-a7cf-00d6099655f1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:42.059980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 557/1103] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:59:41 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:42.067383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:42.080676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:42.081461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:42.081748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:42.082014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:42.082899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:42.083213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:42.084768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:42.109798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 547/1104] 10.222.0.48 () {56 vars in 1133 bytes} [Tue Jul 26 16:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:42.116812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:42.263458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6549266-a381-4941-812d-3867c8a539a4 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:42.319854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:42.320908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:42.321429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:42.321914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:42.387372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:42.427027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:42.428539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:42.583647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 558/1105] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:42.590643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:42.718850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be439c9f-1dca-4019-a91c-5d9dca79bc37 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:42.761322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:42.762100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:42.762470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:42.763371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:42.776766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:42.779006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 16:59:42.799183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:42.801298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:42.803187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 16:59:42.803187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 16:59:42.803187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 16:59:42.803187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:42.803187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 548/1106] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:42.810970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:42.900904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:42.959195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fe174-6b7f-4a3b-8fff-d1d1b45b8736 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:43.030572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:43.031663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:43.032176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:43.034597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:43.034597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:43.034597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:43.037127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:43.058312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 559/1107] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 16:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:43.065520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:43.124735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:43.228573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92320133-c9aa-412b-922a-4d97c41e9c57 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:43.292392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:43.293520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:43.294556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:43.295063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:43.307991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:43.311462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 16:59:43.334272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 16:59:43.336056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 16:59:43.338201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 549/1108] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 16:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 16:59:43.344418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:43.498489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da84b59-1b3a-46ba-9a10-ec7e980cdeb8 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:43.534178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:43.540292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:43.540931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:43.541248 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:43.541520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:43.542284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:43.542593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:43.544658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c359-e036-4f7f-8359-282a98fe0d5c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:43.559281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 560/1109] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 16:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:43.566537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:43.750871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:43.760350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d85de-7cf2-4914-af52-da6e6e535d8a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:43.800944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:43.801749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:43.802089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:43.802431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:43.803309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] RBAC: Adding query filter params (role.id=b9ea0ba5c50947e6ad644321a285dc4a, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=a903f3771a62447d9c06fbeba414e9e3) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:43.803683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:43.805250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304d22c-0fa7-4e0d-a5b7-6cb684abe1ff None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:43.812527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 550/1110] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:43 2022] GET /identity/v3/role_assignments?user.id=a903f3771a62447d9c06fbeba414e9e3&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=b9ea0ba5c50947e6ad644321a285dc4a => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:44.565264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c376fb8-54c7-4ac8-abea-5e20695fcf6c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:44.566056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c376fb8-54c7-4ac8-abea-5e20695fcf6c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:44.566056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c376fb8-54c7-4ac8-abea-5e20695fcf6c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:44.567625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 561/1111] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:44.575169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537deda-4427-436f-9331-951bc71ed670 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:44.576384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537deda-4427-436f-9331-951bc71ed670 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:44.576384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537deda-4427-436f-9331-951bc71ed670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:44.597671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a537deda-4427-436f-9331-951bc71ed670 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:44.598940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a537deda-4427-436f-9331-951bc71ed670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:44.691912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 551/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 16:59:44.717310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c376fb8-54c7-4ac8-abea-5e20695fcf6c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:44.867284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c376fb8-54c7-4ac8-abea-5e20695fcf6c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:44.917656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:44.918321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:44.918598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:44.918866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] PATH_INFO: `/v3/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:44.920426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] RBAC: Authorizing `identity:get_role(role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:44.921867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:44.923276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 562/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:44 2022] GET /identity/v3/roles/55b0174dc86246af94379fcc1ef68040 => generated 223 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:44.933534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537deda-4427-436f-9331-951bc71ed670 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:45.101634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:45.115364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537deda-4427-436f-9331-951bc71ed670 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:45.160137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:45.160808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:45.161125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:45.161403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] PATH_INFO: `/v3/users/53939227a6a84b058eade3a7d5aee5b0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:45.177776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] RBAC: Authorizing `identity:get_user(user_id=53939227a6a84b058eade3a7d5aee5b0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:45.179265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:45.181080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 552/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:44 2022] GET /identity/v3/users/53939227a6a84b058eade3a7d5aee5b0 => generated 286 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:45.187988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:45.261459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:45.315049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b26d7-38db-4ce6-81d9-bcca5a6dffa6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:45.358293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:45.358733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:45.358932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:45.359205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:45.445693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:45.447247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:45.522794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:45.573818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 563/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:45.580912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:45.704816 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d47d-8b36-4d39-9f3d-f3f231e8195a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:45.749032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85015286-460e-4497-94e2-fa5294ddfad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:45.749766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85015286-460e-4497-94e2-fa5294ddfad9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:45.750088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85015286-460e-4497-94e2-fa5294ddfad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:45.750404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85015286-460e-4497-94e2-fa5294ddfad9 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:45.751743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85015286-460e-4497-94e2-fa5294ddfad9 None admin] RBAC: Authorizing `identity:get_project(project_id=ee7c2557de664bfca510590a9110d277)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:45.753348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85015286-460e-4497-94e2-fa5294ddfad9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:45.755217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 553/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:59:45 2022] GET /identity/v3/projects/ee7c2557de664bfca510590a9110d277 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:45.762462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:45.879140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56a25e-14d0-4dda-8939-dd441f120bec None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:45.919101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:45.919819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:45.920189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:45.920549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:45.921461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] RBAC: Adding query filter params (role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=ee7c2557de664bfca510590a9110d277, user.id=53939227a6a84b058eade3a7d5aee5b0) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:45.921818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:45.923350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8586e0c-2165-46ba-8c5d-11c4bae069ed None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:45.929839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 564/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:45 2022] GET /identity/v3/role_assignments?user.id=53939227a6a84b058eade3a7d5aee5b0&scope.project.id=ee7c2557de664bfca510590a9110d277&role.id=55b0174dc86246af94379fcc1ef68040 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:48.178268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7af121-4f80-4aa8-bf20-12dd6b1298af None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:48.178884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7af121-4f80-4aa8-bf20-12dd6b1298af None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:48.178884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7af121-4f80-4aa8-bf20-12dd6b1298af None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:48.180247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 554/1118] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:48.189892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:48.190493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:48.190493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:48.210521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:48.211756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:48.234069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:48.296005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 565/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:48.319776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7af121-4f80-4aa8-bf20-12dd6b1298af None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:48.341420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:48.452810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7af121-4f80-4aa8-bf20-12dd6b1298af None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:48.495537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:48.496208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:48.496492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:48.496778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] PATH_INFO: `/v3/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:48.498926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] RBAC: Authorizing `identity:get_role(role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:48.500351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:48.501864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 555/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:48 2022] GET /identity/v3/roles/55b0174dc86246af94379fcc1ef68040 => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:48.511346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:48.650200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613ebb1-2ec0-4469-a6cd-1f42cbdf950a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:48.688077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:48.688802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:48.689165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:48.689489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] PATH_INFO: `/v3/users/53939227a6a84b058eade3a7d5aee5b0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:48.690846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] RBAC: Authorizing `identity:get_user(user_id=53939227a6a84b058eade3a7d5aee5b0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:48.692384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:48.694195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 566/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:48 2022] GET /identity/v3/users/53939227a6a84b058eade3a7d5aee5b0 => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:48.701967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:48.799608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:48.818797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb19004-47ec-4a03-be3a-610eb1e604f2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:48.868003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f190a-766c-46d9-ab00-62a0be378034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:48.868723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f190a-766c-46d9-ab00-62a0be378034 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:48.869094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f190a-766c-46d9-ab00-62a0be378034 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:48.869415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f190a-766c-46d9-ab00-62a0be378034 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:48.870733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f190a-766c-46d9-ab00-62a0be378034 None admin] RBAC: Authorizing `identity:get_project(project_id=ee7c2557de664bfca510590a9110d277)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:48.873287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f190a-766c-46d9-ab00-62a0be378034 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:48.875133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 556/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:59:48 2022] GET /identity/v3/projects/ee7c2557de664bfca510590a9110d277 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:48.881589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:48.910702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:49.001743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f6f74-17ca-464f-acb1-971f563ad479 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:49.042268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e63e6-e4f9-45ed-91b4-c6fba4f92682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:49.042998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e63e6-e4f9-45ed-91b4-c6fba4f92682 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:49.043333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e63e6-e4f9-45ed-91b4-c6fba4f92682 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:49.043651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e63e6-e4f9-45ed-91b4-c6fba4f92682 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277/users/53939227a6a84b058eade3a7d5aee5b0/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:49.046820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e63e6-e4f9-45ed-91b4-c6fba4f92682 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee7c2557de664bfca510590a9110d277, user_id=53939227a6a84b058eade3a7d5aee5b0, role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:49.048374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e63e6-e4f9-45ed-91b4-c6fba4f92682 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:49.064832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 567/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 16:59:48 2022] PUT /identity/v3/projects/ee7c2557de664bfca510590a9110d277/users/53939227a6a84b058eade3a7d5aee5b0/roles/55b0174dc86246af94379fcc1ef68040 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 16:59:51.256524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e81621-c2bd-46cb-b782-9e76ea24534c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:51.257133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e81621-c2bd-46cb-b782-9e76ea24534c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:51.257133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e81621-c2bd-46cb-b782-9e76ea24534c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:51.258460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 557/1124] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 16:59:51.264264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:51.264850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:51.264850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:51.286564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:51.287784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:51.371541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 568/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:51.397014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e81621-c2bd-46cb-b782-9e76ea24534c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:51.525912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e81621-c2bd-46cb-b782-9e76ea24534c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:51.566795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:51.567467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:51.567751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:51.568028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] PATH_INFO: `/v3/roles/55b0174dc86246af94379fcc1ef68040` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:51.569653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] RBAC: Authorizing `identity:get_role(role_id=55b0174dc86246af94379fcc1ef68040)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:51.571060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:51.572487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 558/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:51 2022] GET /identity/v3/roles/55b0174dc86246af94379fcc1ef68040 => generated 223 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:51.579275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:51.594917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:51.707992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c715c-6861-43d5-a8c2-f2e077cf7942 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:51.750937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:51.751598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:51.751880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:51.752154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] PATH_INFO: `/v3/users/53939227a6a84b058eade3a7d5aee5b0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:51.753432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] RBAC: Authorizing `identity:get_user(user_id=53939227a6a84b058eade3a7d5aee5b0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:51.754921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:51.757024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 569/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 16:59:51 2022] GET /identity/v3/users/53939227a6a84b058eade3a7d5aee5b0 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:51.764617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:51.790808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:51.877789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed1d8-a922-4497-a638-5e416df08dc8 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:51.915780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:51.916407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:51.916691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:51.917073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:51.990395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:51.991862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:52.015549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:52.101734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 559/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 16:59:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 16:59:52.108784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:52.196077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:52.217828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f25c5b-02f3-4012-9455-43ecae53f185 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:52.255288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76206a5-3660-45cb-aaa0-f3b02d7de864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:52.255947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76206a5-3660-45cb-aaa0-f3b02d7de864 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:52.256223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76206a5-3660-45cb-aaa0-f3b02d7de864 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:52.256494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76206a5-3660-45cb-aaa0-f3b02d7de864 None admin] PATH_INFO: `/v3/projects/ee7c2557de664bfca510590a9110d277` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:52.257724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76206a5-3660-45cb-aaa0-f3b02d7de864 None admin] RBAC: Authorizing `identity:get_project(project_id=ee7c2557de664bfca510590a9110d277)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:52.259248 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76206a5-3660-45cb-aaa0-f3b02d7de864 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:52.261015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 570/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 16:59:52 2022] GET /identity/v3/projects/ee7c2557de664bfca510590a9110d277 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:52.268872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:52.385266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef43b2-e4db-4a03-9925-a931cf72b98f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:52.409400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:52.430182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:52.430810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:52.431092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:52.431397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:52.432209 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] RBAC: Adding query filter params (role.id=55b0174dc86246af94379fcc1ef68040, scope.project.id=ee7c2557de664bfca510590a9110d277, user.id=53939227a6a84b058eade3a7d5aee5b0) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 16:59:52.432521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:52.434480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f32ea5-b150-471e-b924-99eb7618a6c9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:52.441865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 560/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 16:59:52 2022] GET /identity/v3/role_assignments?user.id=53939227a6a84b058eade3a7d5aee5b0&scope.project.id=ee7c2557de664bfca510590a9110d277&role.id=55b0174dc86246af94379fcc1ef68040 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:54.575936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4f97-7160-4b09-8506-5772abdaa539 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:54.576542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4f97-7160-4b09-8506-5772abdaa539 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:54.576720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4f97-7160-4b09-8506-5772abdaa539 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:54.578558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 571/1131] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:54.584622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:54.585289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:54.585412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:54.605551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:54.607733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:54.682199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 561/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:54.706174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4f97-7160-4b09-8506-5772abdaa539 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:54.820798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4f97-7160-4b09-8506-5772abdaa539 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:54.860002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163b70-4f2e-44af-bfdc-ff7ce28665dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:54.860674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163b70-4f2e-44af-bfdc-ff7ce28665dc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:54.860988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163b70-4f2e-44af-bfdc-ff7ce28665dc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:54.861260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163b70-4f2e-44af-bfdc-ff7ce28665dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:54.862499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163b70-4f2e-44af-bfdc-ff7ce28665dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:54.863981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2163b70-4f2e-44af-bfdc-ff7ce28665dc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:54.865713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 572/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 16:59:54.872094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:54.989963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857d2ea-49a9-40f3-bc26-b9613543930e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:55.015639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:55.053383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44f4de-987e-402c-8940-012f6f0dcdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:55.054427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44f4de-987e-402c-8940-012f6f0dcdc7 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:55.054906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44f4de-987e-402c-8940-012f6f0dcdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:55.055385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44f4de-987e-402c-8940-012f6f0dcdc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:55.056624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44f4de-987e-402c-8940-012f6f0dcdc7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:55.059380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d44f4de-987e-402c-8940-012f6f0dcdc7 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:55.096311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 562/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 16:59:54 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 16:59:57.401826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5682-4af4-4d95-b4a5-17d3f83c71ca None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:57.402722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5682-4af4-4d95-b4a5-17d3f83c71ca None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:57.402722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f5682-4af4-4d95-b4a5-17d3f83c71ca None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:57.405365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 573/1135] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 16:59:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 16:59:57.411497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:57.412053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:57.412053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:57.431685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 16:59:57.432913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 16:59:57.509630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 563/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 16:59:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 16:59:57.536241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5682-4af4-4d95-b4a5-17d3f83c71ca None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:57.557638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:57.656889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f5682-4af4-4d95-b4a5-17d3f83c71ca None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:57.695895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f8f04-4944-4a36-ad1b-6ceb43e4b3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:57.696563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f8f04-4944-4a36-ad1b-6ceb43e4b3bc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:57.696847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f8f04-4944-4a36-ad1b-6ceb43e4b3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:57.697153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f8f04-4944-4a36-ad1b-6ceb43e4b3bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:57.698389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f8f04-4944-4a36-ad1b-6ceb43e4b3bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:57.699868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f8f04-4944-4a36-ad1b-6ceb43e4b3bc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:57.702511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 574/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 16:59:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 16:59:57.708649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 16:59:57.814766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 16:59:57.822752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033a99d-4b6b-4ebe-aa17-c8ce95a404dc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 16:59:57.863793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1770d3-2aae-4a56-ae10-8e5513f08595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 16:59:57.864479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1770d3-2aae-4a56-ae10-8e5513f08595 None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 16:59:57.864803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1770d3-2aae-4a56-ae10-8e5513f08595 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 16:59:57.865227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1770d3-2aae-4a56-ae10-8e5513f08595 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 16:59:57.866004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1770d3-2aae-4a56-ae10-8e5513f08595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 16:59:57.867538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1770d3-2aae-4a56-ae10-8e5513f08595 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 16:59:57.892482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 564/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 16:59:57 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:00:00.135437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514c0d5-18f1-4369-bc0e-e5adfc24a778 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:00.136060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514c0d5-18f1-4369-bc0e-e5adfc24a778 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:00.136351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514c0d5-18f1-4369-bc0e-e5adfc24a778 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:00.138909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 575/1139] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:00:00.144874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:00.145492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:00.145492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:00.165549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:00.166769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:00.245087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 565/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:00:00.271237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514c0d5-18f1-4369-bc0e-e5adfc24a778 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:00.371726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:00.393143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514c0d5-18f1-4369-bc0e-e5adfc24a778 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:00.448862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:00.450023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:00.451703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:00.452817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:00.466638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:00:00.473073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:00.475012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:00:00.476762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 576/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:00:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:00:00.484275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:00.582366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:00.604754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a98c8-76f7-4080-826a-6a60b0fe2dea None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:00.646341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:00.646879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:00.647161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:00.647438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:00.648187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:00.648548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:00.649936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:00.659777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 566/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:00:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:00.666146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:00.782333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2258-3996-4944-8f81-7f0fbf10d45b None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:00.821726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391f917-4408-4631-9343-5495548f561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:00.822383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391f917-4408-4631-9343-5495548f561a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:00.822976 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391f917-4408-4631-9343-5495548f561a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:00.823251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391f917-4408-4631-9343-5495548f561a None admin] PATH_INFO: `/v3/users/d8b6a0a06714495c913b9ef9bc7100e1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:00.838970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391f917-4408-4631-9343-5495548f561a None admin] RBAC: Authorizing `identity:get_user(user_id=d8b6a0a06714495c913b9ef9bc7100e1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:00.840455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391f917-4408-4631-9343-5495548f561a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:00.842650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 577/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 17:00:00 2022] GET /identity/v3/users/d8b6a0a06714495c913b9ef9bc7100e1 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:00.848836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:00.964689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3b1fd-1438-4e6c-934b-85c818cc352f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:01.006366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:01.006994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:01.007268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:01.007536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:01.025489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:01.088817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:01.090303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:01.207751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 567/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:00:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:00:01.214094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391f917-4408-4631-9343-5495548f561a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:01.217877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:01.332143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391f917-4408-4631-9343-5495548f561a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:01.375037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c772fe1-963c-42df-b51c-c166de175882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:01.375708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c772fe1-963c-42df-b51c-c166de175882 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:01.375990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c772fe1-963c-42df-b51c-c166de175882 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:01.376264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c772fe1-963c-42df-b51c-c166de175882 None admin] PATH_INFO: `/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:01.377551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c772fe1-963c-42df-b51c-c166de175882 None admin] RBAC: Authorizing `identity:get_project(project_id=2bf5c28bd3a340a7b48704c5ed6af944)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:01.379037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c772fe1-963c-42df-b51c-c166de175882 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:01.380831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 578/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 17:00:01 2022] GET /identity/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:01.387471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:01.440114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:01.502549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2e91f-3151-46f0-b676-f042e94eb8bc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:01.543097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:01.543720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:01.544198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:01.544712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:01.545295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=2bf5c28bd3a340a7b48704c5ed6af944, user.id=d8b6a0a06714495c913b9ef9bc7100e1) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:01.545605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:01.547046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d2925-dd8f-46f1-88fc-37655022d368 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:01.553765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 568/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:00:01 2022] GET /identity/v3/role_assignments?user.id=d8b6a0a06714495c913b9ef9bc7100e1&scope.project.id=2bf5c28bd3a340a7b48704c5ed6af944&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:03.762089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad9e6f-b02f-4c71-ab83-cfcb7fcc8995 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:03.763153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad9e6f-b02f-4c71-ab83-cfcb7fcc8995 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:03.763153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad9e6f-b02f-4c71-ab83-cfcb7fcc8995 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:03.764735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 579/1147] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:00:03.770900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:03.771428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:03.771428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:03.790812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:03.792054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:03.874761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 569/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:00:03.898243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad9e6f-b02f-4c71-ab83-cfcb7fcc8995 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:03.982131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:04.021111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad9e6f-b02f-4c71-ab83-cfcb7fcc8995 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:04.062752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:04.063413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:04.063693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:04.063968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:04.077021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:00:04.082444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:04.084370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:00:04.086170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 580/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:00:04.090514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:04.137604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:04.209556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc43171-397d-41ce-93cf-e4e89cd6c1d9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:04.252294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:04.253008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:04.253351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:04.253721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:04.254522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:04.254916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:04.256385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:04.266899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 570/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:04.273867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:04.391970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfd602-3812-4aa8-a420-8659c49c4baf None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:04.427879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:04.436320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:04.437009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:04.437305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:04.437583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] PATH_INFO: `/v3/users/d8b6a0a06714495c913b9ef9bc7100e1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:04.438867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] RBAC: Authorizing `identity:get_user(user_id=d8b6a0a06714495c913b9ef9bc7100e1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:04.440347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:04.442069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 581/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 17:00:04 2022] GET /identity/v3/users/d8b6a0a06714495c913b9ef9bc7100e1 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:04.448290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:04.568385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:04.572036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e964a2-03f0-4843-a6db-461a1d553499 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:04.613399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa6784-a71f-4fc5-bd76-621b43c956ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:04.614057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa6784-a71f-4fc5-bd76-621b43c956ef None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:04.614344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa6784-a71f-4fc5-bd76-621b43c956ef None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:04.614614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa6784-a71f-4fc5-bd76-621b43c956ef None admin] PATH_INFO: `/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:04.615811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa6784-a71f-4fc5-bd76-621b43c956ef None admin] RBAC: Authorizing `identity:get_project(project_id=2bf5c28bd3a340a7b48704c5ed6af944)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:04.617351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa6784-a71f-4fc5-bd76-621b43c956ef None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:04.619120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 571/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 17:00:04 2022] GET /identity/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:04.627369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:04.751272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1497a18-4065-49c2-9e41-8b02e6040cae None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:04.793400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78019-0807-4773-b5c7-3aa8cbcbf1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:04.794520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78019-0807-4773-b5c7-3aa8cbcbf1dd None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:04.794520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78019-0807-4773-b5c7-3aa8cbcbf1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:04.795000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78019-0807-4773-b5c7-3aa8cbcbf1dd None admin] PATH_INFO: `/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944/users/d8b6a0a06714495c913b9ef9bc7100e1/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:04.797279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78019-0807-4773-b5c7-3aa8cbcbf1dd None admin] RBAC: Authorizing `identity:create_grant(project_id=2bf5c28bd3a340a7b48704c5ed6af944, user_id=d8b6a0a06714495c913b9ef9bc7100e1, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:04.798795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78019-0807-4773-b5c7-3aa8cbcbf1dd None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:04.815706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 582/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 17:00:04 2022] PUT /identity/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944/users/d8b6a0a06714495c913b9ef9bc7100e1/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:00:07.874830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97789-60ee-4a95-bc4e-952477e9b49d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:07.875400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97789-60ee-4a95-bc4e-952477e9b49d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:07.875400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97789-60ee-4a95-bc4e-952477e9b49d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:07.876803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 572/1154] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:00:07.882942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:07.883513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:07.883513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:07.904574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:07.905852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:07.966843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:07.984861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 583/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:00:08.009064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e97789-60ee-4a95-bc4e-952477e9b49d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:08.140130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e97789-60ee-4a95-bc4e-952477e9b49d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:08.181029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:08.181694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:08.181975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:08.182252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:08.195285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:00:08.201008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:08.202648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:00:08.203565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 573/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:00:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:00:08.209582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:08.334232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c6877-3b6e-448d-8298-b274ecfac326 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:08.382038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:08.382679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:08.382960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:08.383233 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:08.383967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:08.384269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:08.385729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:08.396114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 584/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:00:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:08.405879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:08.440084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:08.540779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b3272-d69c-47ea-99d7-446176c7ef7c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:08.579317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:08.580027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:08.580353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:08.580686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] PATH_INFO: `/v3/users/d8b6a0a06714495c913b9ef9bc7100e1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:08.582068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=d8b6a0a06714495c913b9ef9bc7100e1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:08.583585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:08.585429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 574/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 17:00:08 2022] GET /identity/v3/users/d8b6a0a06714495c913b9ef9bc7100e1 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:08.594323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:08.652905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:08.712909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78003d30-80a6-4a58-be47-505614aa490a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:08.756291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:08.757283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:08.757566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:08.757840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:08.847227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:08.848840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:08.907719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:08.968230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 585/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:00:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:00:08.974630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:09.086416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:09.099468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4b5b1-4737-45aa-8252-44f9472dc6f2 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:09.163199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f44953-d7f3-40eb-8fa0-902231d6a6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:09.163776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44953-d7f3-40eb-8fa0-902231d6a6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:09.164055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44953-d7f3-40eb-8fa0-902231d6a6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:09.164328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f44953-d7f3-40eb-8fa0-902231d6a6b9 None admin] PATH_INFO: `/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:09.165625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f44953-d7f3-40eb-8fa0-902231d6a6b9 None admin] RBAC: Authorizing `identity:get_project(project_id=2bf5c28bd3a340a7b48704c5ed6af944)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:09.167096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f44953-d7f3-40eb-8fa0-902231d6a6b9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:09.169174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 575/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 17:00:08 2022] GET /identity/v3/projects/2bf5c28bd3a340a7b48704c5ed6af944 => generated 311 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:09.175928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:09.299674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc9368-6caf-4844-88f1-6c680111e4f9 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:09.340378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:09.341253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:09.341637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:09.342016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:09.342920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=2bf5c28bd3a340a7b48704c5ed6af944, user.id=d8b6a0a06714495c913b9ef9bc7100e1) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:09.343329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:09.344914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b56d0f-70ac-400f-9c8b-adeef727ba18 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:09.351844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 586/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:00:09 2022] GET /identity/v3/role_assignments?user.id=d8b6a0a06714495c913b9ef9bc7100e1&scope.project.id=2bf5c28bd3a340a7b48704c5ed6af944&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:11.584333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a7915-63f3-49ee-9867-42a0b0c0d4ec None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:11.584991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a7915-63f3-49ee-9867-42a0b0c0d4ec None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:11.584991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a7915-63f3-49ee-9867-42a0b0c0d4ec None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:11.586545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 576/1162] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:00:11.594117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:11.594706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:11.594706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:11.619391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:11.620628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:11.632630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:11.705856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 587/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:00:11.729896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a7915-63f3-49ee-9867-42a0b0c0d4ec None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:11.896301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a7915-63f3-49ee-9867-42a0b0c0d4ec None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:11.937285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:11.938002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:11.938329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:11.938638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:11.980398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:12.033642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:12.035417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:12.172911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 577/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:00:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:00:12.182931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:12.374737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff769-52b9-46a4-b546-4a080d49b583 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:12.422812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:12.423492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:12.423772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:12.424329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:12.432028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:12.433659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-590877d0-73e6-47d8-81c4-026d74f505d7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:12.448842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:12.450382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:00:12.451270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 588/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 17:00:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:00:12.458414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:12.554180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:12.609783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8b345-183a-4d4a-a0b3-0d9e7a244d8e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:12.656421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:12.656991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:12.657269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:12.657550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:12.658269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:12.658570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:12.661683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52babf-9dcb-4d86-8f29-56ca0b949363 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:12.676570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 578/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Tue Jul 26 17:00:12 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:14.882899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589ff35-01d1-4559-859e-e5ccbfddeba8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:14.883511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589ff35-01d1-4559-859e-e5ccbfddeba8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:14.883511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589ff35-01d1-4559-859e-e5ccbfddeba8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:14.884838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 589/1167] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:00:14.891407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf504b-08c0-481a-8232-999b5e0dfb7a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:14.891948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf504b-08c0-481a-8232-999b5e0dfb7a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:14.892119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf504b-08c0-481a-8232-999b5e0dfb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:14.912041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-78bf504b-08c0-481a-8232-999b5e0dfb7a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:14.913379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-78bf504b-08c0-481a-8232-999b5e0dfb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:14.998081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 579/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:00:15.022579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589ff35-01d1-4559-859e-e5ccbfddeba8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:15.027419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:15.161266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589ff35-01d1-4559-859e-e5ccbfddeba8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:15.198701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bab92c-60a3-4c35-bfe8-d9609560801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:15.199396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bab92c-60a3-4c35-bfe8-d9609560801a None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:15.199711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bab92c-60a3-4c35-bfe8-d9609560801a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:15.200021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bab92c-60a3-4c35-bfe8-d9609560801a None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:15.200808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bab92c-60a3-4c35-bfe8-d9609560801a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:15.202556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bab92c-60a3-4c35-bfe8-d9609560801a None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:15.220942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 590/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Tue Jul 26 17:00:15 2022] POST /identity/v3/domains => generated 267 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:00:17.345859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fef48-260d-4311-8baa-736b9c2d079b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:17.346430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fef48-260d-4311-8baa-736b9c2d079b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:17.346627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5fef48-260d-4311-8baa-736b9c2d079b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:17.349241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 580/1170] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:00:17.357818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:17.358400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:17.358505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:17.378441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:17.379660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:17.451904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:17.464032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 591/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 17:00:17.489372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fef48-260d-4311-8baa-736b9c2d079b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:17.557681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:17.608261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5fef48-260d-4311-8baa-736b9c2d079b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:17.648825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03bf67-2fed-44a4-ae6d-b15836c16ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:17.649518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03bf67-2fed-44a4-ae6d-b15836c16ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:17.649796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03bf67-2fed-44a4-ae6d-b15836c16ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:17.650078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03bf67-2fed-44a4-ae6d-b15836c16ef8 None admin] PATH_INFO: `/v3/domains/cedd915b770f49b8b815cfa2518c4043` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:17.662400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03bf67-2fed-44a4-ae6d-b15836c16ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cedd915b770f49b8b815cfa2518c4043)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:17.663490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03bf67-2fed-44a4-ae6d-b15836c16ef8 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:17.665297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 581/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Tue Jul 26 17:00:17 2022] GET /identity/v3/domains/cedd915b770f49b8b815cfa2518c4043 => generated 267 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:17.671984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:17.792727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c229-5fe9-4ca2-afc0-be04a4859653 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:17.833017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb807e4a-4190-476d-8263-bb5de4f77dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:17.833713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb807e4a-4190-476d-8263-bb5de4f77dce None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:17.834035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb807e4a-4190-476d-8263-bb5de4f77dce None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:17.834351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb807e4a-4190-476d-8263-bb5de4f77dce None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:17.835114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb807e4a-4190-476d-8263-bb5de4f77dce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:17.836639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb807e4a-4190-476d-8263-bb5de4f77dce None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:17.868805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 592/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 17:00:17 2022] POST /identity/v3/projects => generated 361 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:00:20.161607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175929-629f-488f-93cb-2f726fffed6c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:20.162421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175929-629f-488f-93cb-2f726fffed6c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:20.162578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175929-629f-488f-93cb-2f726fffed6c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:20.164006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 582/1174] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:00:20.171277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:20.172397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:20.172397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:20.194032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:20.195313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:20.253836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:20.287415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 593/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:00:20.313574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97175929-629f-488f-93cb-2f726fffed6c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:20.472675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97175929-629f-488f-93cb-2f726fffed6c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:20.508595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:20.538409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5da7fd-ac18-46bb-8f9f-10c1b5ff9842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:20.539206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da7fd-ac18-46bb-8f9f-10c1b5ff9842 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:20.539493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da7fd-ac18-46bb-8f9f-10c1b5ff9842 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:20.540018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da7fd-ac18-46bb-8f9f-10c1b5ff9842 None admin] PATH_INFO: `/v3/domains/cedd915b770f49b8b815cfa2518c4043` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:20.541502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5da7fd-ac18-46bb-8f9f-10c1b5ff9842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cedd915b770f49b8b815cfa2518c4043)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:20.542980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5da7fd-ac18-46bb-8f9f-10c1b5ff9842 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:20.549010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 583/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Tue Jul 26 17:00:20 2022] GET /identity/v3/domains/cedd915b770f49b8b815cfa2518c4043 => generated 267 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:20.553934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:20.687139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650bd86-9ce5-4b95-9d62-b447ce7e4a23 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:20.727645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9246a-a1c1-42af-abc9-cda3bdde141f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:20.728191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9246a-a1c1-42af-abc9-cda3bdde141f None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:20.728472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9246a-a1c1-42af-abc9-cda3bdde141f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:20.728748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9246a-a1c1-42af-abc9-cda3bdde141f None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:20.729481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9246a-a1c1-42af-abc9-cda3bdde141f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:20.730965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9246a-a1c1-42af-abc9-cda3bdde141f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:20.757842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 594/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 17:00:20 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:00:22.997605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c1628-2c7c-484a-8ce9-5ac7f9ce7f55 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:22.998196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c1628-2c7c-484a-8ce9-5ac7f9ce7f55 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:22.998444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c1628-2c7c-484a-8ce9-5ac7f9ce7f55 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:23.000040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 584/1178] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:00:23.006084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:23.006609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:23.006609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:23.026296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:23.027516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:23.060822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:23.115670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 595/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 17:00:23.142310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c1628-2c7c-484a-8ce9-5ac7f9ce7f55 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:23.290083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c1628-2c7c-484a-8ce9-5ac7f9ce7f55 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:23.367061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:23.368100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:23.368559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:23.369045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:23.390886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:00:23.401043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:23.402926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:23.405792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:00:23.407031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 585/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:00:23.414240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:23.538405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c24d6a-4c59-42a0-b682-00c440d229f1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:23.579362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:23.580015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:23.580298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:23.580571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:23.581341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:23.581645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:23.583090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:23.593438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 596/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:23.600362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:23.699173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:23.764171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b67f9-d3bd-43cb-a927-dc7bcc0e4c9d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:23.803706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:23.804480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:23.804842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:23.805276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] PATH_INFO: `/v3/users/3549799948a44de0b6177a1ed05093ed` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:23.824442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] RBAC: Authorizing `identity:get_user(user_id=3549799948a44de0b6177a1ed05093ed)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:23.826072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:23.827889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 586/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 17:00:23 2022] GET /identity/v3/users/3549799948a44de0b6177a1ed05093ed => generated 311 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:23.837144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:23.866559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:23.959332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2f71d-8ba7-4095-bfac-9f6aa2a942c3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:24.004730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:24.005745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:24.006271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:24.006811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:24.141870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:24.144588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:24.193369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:24.355449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 597/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:00:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:00:24.365198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:24.494304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123c83c-edcf-4463-a8bb-1a3fe0708096 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:24.497832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:24.537669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ba144-f80b-4b37-9476-598ce7d8a717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:24.538929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ba144-f80b-4b37-9476-598ce7d8a717 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:24.539173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ba144-f80b-4b37-9476-598ce7d8a717 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:24.539369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ba144-f80b-4b37-9476-598ce7d8a717 None admin] PATH_INFO: `/v3/projects/a66750c1191748e68d2fa03c73793f2f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:24.540906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ba144-f80b-4b37-9476-598ce7d8a717 None admin] RBAC: Authorizing `identity:get_project(project_id=a66750c1191748e68d2fa03c73793f2f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:24.542610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ba144-f80b-4b37-9476-598ce7d8a717 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:24.545208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 587/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 17:00:24 2022] GET /identity/v3/projects/a66750c1191748e68d2fa03c73793f2f => generated 361 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:24.551685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:24.670820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac0143-59a1-42f1-98eb-cfcc31bbcb21 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:24.699602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:24.713715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:24.714510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:24.714888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:24.715269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:24.716390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=a66750c1191748e68d2fa03c73793f2f, user.id=3549799948a44de0b6177a1ed05093ed) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:24.716846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:24.718488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b058c-2d43-414e-b6d0-4b7cda81ab1d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:24.725204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 598/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:00:24 2022] GET /identity/v3/role_assignments?user.id=3549799948a44de0b6177a1ed05093ed&scope.project.id=a66750c1191748e68d2fa03c73793f2f&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:26.952038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df84894-0b8d-4d64-a944-980938c48aaa None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:26.952631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df84894-0b8d-4d64-a944-980938c48aaa None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:26.952631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df84894-0b8d-4d64-a944-980938c48aaa None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:26.954017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 588/1186] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:00:26.960032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:26.960650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:26.960650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:26.979709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:26.981018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:27.058166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 599/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 17:00:27.081779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df84894-0b8d-4d64-a944-980938c48aaa None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:27.203322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df84894-0b8d-4d64-a944-980938c48aaa None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:27.243484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:27.244151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:27.244435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:27.244710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:27.257999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:00:27.263593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:27.265398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:00:27.266330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 589/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:00:27.275445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:27.433394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a799a-709e-4036-86c2-81572d64e061 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:27.454277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:27.475134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:27.475776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:27.476060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:27.476334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:27.477190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:27.477497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:27.478968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:27.489710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 600/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:27.495875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:27.539712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:27.620157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30910c4b-c3c5-41f2-85fa-2829fdf35a39 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:27.660582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:27.661362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:27.661681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:27.662036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] PATH_INFO: `/v3/users/3549799948a44de0b6177a1ed05093ed` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:27.663419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] RBAC: Authorizing `identity:get_user(user_id=3549799948a44de0b6177a1ed05093ed)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:27.664948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:27.666856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 590/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 17:00:27 2022] GET /identity/v3/users/3549799948a44de0b6177a1ed05093ed => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:27.674315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:27.791712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f416f-b932-4c92-8fbc-af38f24da1d3 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:27.833447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d6139-a049-49c2-94e2-66f2dc8139e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:27.834156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d6139-a049-49c2-94e2-66f2dc8139e2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:27.834474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d6139-a049-49c2-94e2-66f2dc8139e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:27.834785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d6139-a049-49c2-94e2-66f2dc8139e2 None admin] PATH_INFO: `/v3/projects/a66750c1191748e68d2fa03c73793f2f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:27.836099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d6139-a049-49c2-94e2-66f2dc8139e2 None admin] RBAC: Authorizing `identity:get_project(project_id=a66750c1191748e68d2fa03c73793f2f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:27.837713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d6139-a049-49c2-94e2-66f2dc8139e2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:27.839578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 601/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 17:00:27 2022] GET /identity/v3/projects/a66750c1191748e68d2fa03c73793f2f => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:27.847050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:27.975360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:27.978885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42452cc9-7d11-4d1a-9297-81af3e1e91ca None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:28.022768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883915b2-b547-468c-8c5d-1891f096c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:28.023474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883915b2-b547-468c-8c5d-1891f096c7d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:28.023755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883915b2-b547-468c-8c5d-1891f096c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:28.024032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883915b2-b547-468c-8c5d-1891f096c7d2 None admin] PATH_INFO: `/v3/projects/a66750c1191748e68d2fa03c73793f2f/users/3549799948a44de0b6177a1ed05093ed/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:28.026208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883915b2-b547-468c-8c5d-1891f096c7d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a66750c1191748e68d2fa03c73793f2f, user_id=3549799948a44de0b6177a1ed05093ed, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:28.027703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883915b2-b547-468c-8c5d-1891f096c7d2 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:28.042845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 591/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 17:00:27 2022] PUT /identity/v3/projects/a66750c1191748e68d2fa03c73793f2f/users/3549799948a44de0b6177a1ed05093ed/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:00:30.705723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4167b-f0c4-49ce-91bf-3877e1fb7df0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:30.706383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4167b-f0c4-49ce-91bf-3877e1fb7df0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:30.706383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4167b-f0c4-49ce-91bf-3877e1fb7df0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:30.707568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 602/1193] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:00:30.713473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:30.714231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:30.714231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:30.736670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:00:30.737951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:00:30.819420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 592/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:00:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 17:00:30.848001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4167b-f0c4-49ce-91bf-3877e1fb7df0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:30.910836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:30.976205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4167b-f0c4-49ce-91bf-3877e1fb7df0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:31.020160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:31.020829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:31.021156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:31.021433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:31.034145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:00:31.039460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:00:31.041264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:00:31.043127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 603/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:00:31.050050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:31.159245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:31.172242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78581e9d-4ca4-4987-b9a7-b3e7ffc13aa0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:31.214042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:31.214683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:31.214966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:31.215242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:31.216541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:31.216849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:31.218428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:31.228726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 593/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:31.236556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:31.427540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd6934-ac80-4aeb-a58d-b4d0337e71bb None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:31.443662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:31.475713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:31.476412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:31.476718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:31.477042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] PATH_INFO: `/v3/users/3549799948a44de0b6177a1ed05093ed` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:31.478631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] RBAC: Authorizing `identity:get_user(user_id=3549799948a44de0b6177a1ed05093ed)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:31.480276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:31.483012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 604/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Tue Jul 26 17:00:31 2022] GET /identity/v3/users/3549799948a44de0b6177a1ed05093ed => generated 311 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:31.489420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:31.621447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f7153-ee87-40c4-b46d-bb9aacd31a20 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:31.662503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:31.663148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:31.663461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:31.663735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:31.690600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:31.749853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:31.751266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:31.871276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 594/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:00:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:00:31.878372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:32.023811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db88c02-d8a7-4790-90c3-34493bdc41cc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:32.065425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32f716-d7cb-4c63-b631-b46d38d72722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:32.066126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32f716-d7cb-4c63-b631-b46d38d72722 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:32.066445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32f716-d7cb-4c63-b631-b46d38d72722 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:32.066768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32f716-d7cb-4c63-b631-b46d38d72722 None admin] PATH_INFO: `/v3/projects/a66750c1191748e68d2fa03c73793f2f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:32.068031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32f716-d7cb-4c63-b631-b46d38d72722 None admin] RBAC: Authorizing `identity:get_project(project_id=a66750c1191748e68d2fa03c73793f2f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:32.069702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32f716-d7cb-4c63-b631-b46d38d72722 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:32.071529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 605/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Tue Jul 26 17:00:31 2022] GET /identity/v3/projects/a66750c1191748e68d2fa03c73793f2f => generated 361 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:00:32.078817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:00:32.143990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:32.199256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68678454-1b5c-4175-ba41-187c9b6069df None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:00:32.241107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:00:32.241740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:00:32.242020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:00:32.242291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:00:32.243098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=a66750c1191748e68d2fa03c73793f2f, user.id=3549799948a44de0b6177a1ed05093ed) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:00:32.243446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:00:32.244915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305ebe3-22af-4b35-b748-e12a41ba09d6 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:00:32.252623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 595/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:00:32 2022] GET /identity/v3/role_assignments?user.id=3549799948a44de0b6177a1ed05093ed&scope.project.id=a66750c1191748e68d2fa03c73793f2f&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:00:42.060536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=78 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:00:42.249527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=42 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:35.974146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774cc23-7c3a-418b-97f7-261963509378 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:35.974146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774cc23-7c3a-418b-97f7-261963509378 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:35.974146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774cc23-7c3a-418b-97f7-261963509378 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:35.974146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 606/1201] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:01:35.982077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:35.982667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:35.982846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:36.006530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:01:36.008156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:01:36.132988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 596/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:01:36.151000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774cc23-7c3a-418b-97f7-261963509378 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:36.358468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774cc23-7c3a-418b-97f7-261963509378 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:36.427445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:36.428496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:36.428928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:36.429446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:36.443159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:36.444657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:36.467328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:36.469316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:36.471310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:36.471310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:36.471310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:36.471310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:36.473825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 607/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:36.480919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:36.664104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6815c-e10c-4c3a-ba77-0748058f8f13 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:36.707972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:36.717246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:36.718447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:36.718734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:36.719036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:36.719817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:36.720136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:36.721918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f301c-c20d-4255-a9c3-d2675aad2056 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:36.738059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 597/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:36.745041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:36.876301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:36.975904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2b4b5-82ef-4f97-9a67-f1e29f12a361 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:37.058679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ead8cd-d546-43e4-aeec-2ea5d1221bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:37.060758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead8cd-d546-43e4-aeec-2ea5d1221bac None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:37.061527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead8cd-d546-43e4-aeec-2ea5d1221bac None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:37.062616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ead8cd-d546-43e4-aeec-2ea5d1221bac None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:37.064205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ead8cd-d546-43e4-aeec-2ea5d1221bac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:37.067220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ead8cd-d546-43e4-aeec-2ea5d1221bac None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:37.115102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 608/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Tue Jul 26 17:01:36 2022] POST /identity/v3/users => generated 251 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:01:40.071184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfeaa4-fb54-499b-9488-825565b3d1d2 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:40.071915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfeaa4-fb54-499b-9488-825565b3d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:40.072268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfeaa4-fb54-499b-9488-825565b3d1d2 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:40.075796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 598/1206] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:01:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:01:40.084880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:40.084880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:40.084880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:40.120320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:01:40.123213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:01:40.268757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 609/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:01:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jul 26 17:01:40.319702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfeaa4-fb54-499b-9488-825565b3d1d2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:40.518673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfeaa4-fb54-499b-9488-825565b3d1d2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:40.582229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:40.583308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:40.583779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:40.584254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:40.598604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:01:40.605774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:40.607369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 17:01:40.608238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:40.609320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 599/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 17:01:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:40.619825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:40.704841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:40.831701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0550296-4461-4460-81aa-5f9c7a83382b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:40.891394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:40.892470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:40.892953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:40.893461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:40.894752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:40.895282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:40.897973 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:40.913844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 610/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 17:01:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:40.920563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:41.019274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:41.094955 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968abb35-0a50-4943-833a-c3edab3b2ba1 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:41.150291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:41.151350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:41.151891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:41.152410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:41.171198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:41.173405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:41.196597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:41.198438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:41.200179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:41.200179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:41.200179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:41.200179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:41.206473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 600/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:01:41.208483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:41.377619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:41.382562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7938-2800-4751-b583-c98f66669de1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:41.446741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:41.447498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:41.447882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:41.448200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:41.449045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:41.449451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:41.452059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:41.466400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 611/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:41.473228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:41.619793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7606725-54f3-4f52-a23c-6587cf2984bb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:41.622351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:41.677009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:41.677836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:41.678206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:41.678535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:41.690007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:41.692216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 17:01:41.709377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:41.710629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:01:41.711749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:41.717507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 601/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:41.721888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:41.888506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc376c-f7be-4244-ae6e-fa662ed3ffbe None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:41.939524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:41.940219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:41.940568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:41.940900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:41.941859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:41.942224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:41.943810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:41.977021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 612/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Tue Jul 26 17:01:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:41.979392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:42.133544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5197c-b64a-4111-b297-890fa24c01ab None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:42.193219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:42.193800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:42.194123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:42.194409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:42.239244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:42.298178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:42.300138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:42.472367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 602/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:01:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:01:42.474227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:42.520051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:42.625665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bbd48-9391-4013-9876-27ba1b25ccc7 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:42.671469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:42.672533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:42.673066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:42.673549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:42.689717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:42.691639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:42.711456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:42.713720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:42.714698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:42.714698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:42.714698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:42.714698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:42.714698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 613/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:42.730562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:42.855635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:42.944896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9fa18-2a6c-4aae-b08a-81711584933c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:43.005190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:43.006477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:43.007236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:43.007999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:43.009625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:43.010263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:43.012950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:43.044873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 603/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:43.053458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:43.226034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:43.285085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f0314-abe4-4539-bdea-dec1fac47533 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:43.357000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:43.357000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:43.357000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:43.357000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:43.371348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:43.375040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 17:01:43.400052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:43.401191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:43.402117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 614/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 17:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:43.410680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:43.613428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a181262-9f4d-418f-9764-aa3c1653f1f3 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:43.663528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:43.689747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:43.691231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:43.691753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:43.692229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:43.693770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:43.694342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:43.697357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5d190-c589-4e65-a680-0dac1ac3f10c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:43.727110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 604/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 17:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:43.739841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:43.966402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a65b58-57e9-4a50-9aca-8dee2cbf05c1 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:44.023473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:44.035935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:44.037253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:44.037836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:44.038325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:44.039816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=6dad20c291c5440ca6f7c89a3effe871) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:44.040276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:44.043157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365e15-7007-4bae-aba7-c693d6ed3412 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:44.056750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 615/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:01:43 2022] GET /identity/v3/role_assignments?user.id=6dad20c291c5440ca6f7c89a3effe871&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:46.980591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb11c-8a16-4e14-90d1-7f4c33299674 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:46.982201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb11c-8a16-4e14-90d1-7f4c33299674 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:46.982201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb11c-8a16-4e14-90d1-7f4c33299674 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:46.982963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 605/1220] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:01:46.989513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:46.990360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:46.990565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:47.010860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:01:47.012184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:01:47.097847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 616/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:01:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:01:47.131435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb11c-8a16-4e14-90d1-7f4c33299674 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:47.306692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb11c-8a16-4e14-90d1-7f4c33299674 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:47.359959 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:47.360563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:47.360889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:47.361194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:47.375738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:01:47.381617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:47.383904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:47.385226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 606/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 17:01:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:47.390967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:47.541890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4c60f-7e7a-4b32-807e-012b1ae5f572 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:47.579928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:47.599063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:47.600164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:47.600674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:47.601246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:47.602641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:47.603213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:47.606526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:47.622993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 617/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 17:01:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:47.632423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:47.695076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:47.788787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9135c443-fd56-4c73-8b2b-cc3e1998392c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:47.841440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:47.841865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:47.842319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:47.842827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:47.861707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:47.863819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:47.888007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:47.889992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:47.891819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:47.891819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:47.891819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:47.891819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:47.895991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 607/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:01:47.897812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:48.064320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441e822-31a0-4ecd-a7af-44c720b51df2 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:48.127636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:48.128287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:48.128632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:48.129060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:48.130265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:48.130716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:48.132461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:48.155167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 618/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:48.164291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:48.342061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:48.353297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f12687-0987-45d2-90ab-ce7757d68c27 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:48.409196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:48.410553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:48.411094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:48.411571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:48.424892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:48.426662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 17:01:48.444114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:48.445425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:48.447835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 608/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:01:48.455288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:48.529386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:48.628093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04315d-89bf-47d4-a01d-d84d87e3b33f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:48.677463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:48.678404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:48.678857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:48.679328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:48.680834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:48.681416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:48.685008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:48.727246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 619/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Tue Jul 26 17:01:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:48.729235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:48.921301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd4011-3f7d-414e-b769-c0b474eba25d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:48.989005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:48.990024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:48.990524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:48.991011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:49.013618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:49.016526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:49.045872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:49.047683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:49.049606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:49.049606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:49.049606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:49.049606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:49.049606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 609/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:49.058027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:49.216547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a114fe5-16a4-472f-a806-ca6c3b9f496e None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:49.221258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:49.269034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c518257-e775-4208-a146-b66685c09874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:49.270026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c518257-e775-4208-a146-b66685c09874 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:49.270525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c518257-e775-4208-a146-b66685c09874 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:49.270996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c518257-e775-4208-a146-b66685c09874 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:49.272328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c518257-e775-4208-a146-b66685c09874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:49.272837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c518257-e775-4208-a146-b66685c09874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:49.275491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c518257-e775-4208-a146-b66685c09874 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:49.292166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 620/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:49.300012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:49.315106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:49.470119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da24930-b898-44ea-9a3c-85788aa30a07 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:49.527602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:49.528879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:49.529235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:49.529622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:49.539635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:49.541771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 17:01:49.562077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:49.564258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:01:49.566115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:49.576422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 610/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 17:01:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:49.577581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c518257-e775-4208-a146-b66685c09874 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:49.763087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c518257-e775-4208-a146-b66685c09874 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:49.828083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:49.829168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:49.829652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:49.830191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:49.831645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:49.832213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:49.835001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e432b-84e7-4930-8d6f-26b181b644e6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:49.865708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 621/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 17:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:49.867459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:49.969084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:50.029442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e261ea62-365d-47a3-80d5-68523661350e None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:50.095991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb481b-1707-4fd7-bd3c-1345282cd667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:50.097109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb481b-1707-4fd7-bd3c-1345282cd667 None admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:50.097599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb481b-1707-4fd7-bd3c-1345282cd667 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:50.098093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb481b-1707-4fd7-bd3c-1345282cd667 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/6dad20c291c5440ca6f7c89a3effe871/roles/df5e2fcd53214d0ca08b0286b303727c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:50.127722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb481b-1707-4fd7-bd3c-1345282cd667 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=6dad20c291c5440ca6f7c89a3effe871, role_id=df5e2fcd53214d0ca08b0286b303727c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:50.129424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb481b-1707-4fd7-bd3c-1345282cd667 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:50.150487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 611/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 17:01:49 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/6dad20c291c5440ca6f7c89a3effe871/roles/df5e2fcd53214d0ca08b0286b303727c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:01:52.603170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf445293-7f08-4ba0-b028-a8a981c54556 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:52.603839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf445293-7f08-4ba0-b028-a8a981c54556 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:52.603839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf445293-7f08-4ba0-b028-a8a981c54556 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:52.605191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 622/1233] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:01:52.612699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:52.613286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:52.613286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:52.635206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:01:52.636435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:01:52.721762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 612/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:01:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:01:52.755249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf445293-7f08-4ba0-b028-a8a981c54556 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:52.787067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:52.898568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf445293-7f08-4ba0-b028-a8a981c54556 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:52.942011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:52.942677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:52.942957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:52.943227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:52.957096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:01:52.962238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:52.963950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:52.966265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 623/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Tue Jul 26 17:01:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:52.970378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:53.040651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:53.137262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9cf1d-cb45-42bd-aff0-3a6af1ae1f93 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:53.194696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:53.195431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:53.195758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:53.196092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:53.196930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:53.197323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:53.199034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:53.211558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 613/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Tue Jul 26 17:01:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:53.219559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:53.391287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:53.422243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a286053-2271-4553-9aa2-7c45b4c00d74 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:53.480558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:53.482063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:53.482398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:53.482702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:53.501502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:53.503091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:53.541835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:53.543674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:53.545528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:53.545528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:53.545528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:53.545528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:53.545528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 624/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:01:53.552472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:53.754254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848863de-f7a3-4485-a353-1de4364427cb None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:53.770780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:53.813132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:53.813914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:53.814291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:53.814658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:53.815513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:53.815904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:53.817533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:53.836826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 614/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:53.846066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:54.051261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cb6f1-c8d2-48b1-a264-a4aeb39a8fbc None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:54.103958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:54.104928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:54.105286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:54.105619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:54.110389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:54.118527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:54.120263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 17:01:54.140999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:54.143481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:54.145807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 625/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:01:54.152866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:54.350047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d2db-4195-422c-bb31-cb8c8332e768 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:54.398213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:54.399140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:54.399589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:54.400042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:54.401142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:54.405021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:54.405021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:54.440288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 615/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Tue Jul 26 17:01:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:54.448358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:54.632525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b88f1-7623-4025-932a-9d7bfaeb9227 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:54.697666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:54.698919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:54.700123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:54.700657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:54.816356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:54.826448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:54.828363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:55.042387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 626/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:01:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:01:55.044871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:55.103994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:55.252489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8703d20-f04d-4367-a02b-60a3b57c685d None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:55.332803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:55.334176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:55.334670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:55.335243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:55.353834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:55.357666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:55.379953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:55.382531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:55.384805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:55.384805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:55.384805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:55.384805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:55.384805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 616/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:55.392287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:55.548984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:55.557245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd71da-c4c1-4987-9f8d-0222ac299781 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:55.616360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:55.617442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:55.617961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:55.618454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:55.619810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:55.620357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:55.623695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:55.648950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 627/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:55.659599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:55.814487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:55.850790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971bd0a-0061-4e62-b638-a9fc83d1a30f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:55.909988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:55.911007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:55.911481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:55.912070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:55.923524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:55.925602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 17:01:55.939628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:55.941799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:01:55.943549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 617/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 17:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:55.951575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:56.121769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e7495-0d74-4290-866e-9b508507ae8d None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:56.191298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:56.191902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:56.192278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:56.192605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:56.193586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:56.194023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:56.195808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe3373-0dec-419c-8e1a-57ff70d2df90 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:56.216380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 628/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 17:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:56.224525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:56.363739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40cef8-7c64-4ab7-83bc-f4f97755c665 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:56.415368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:56.416017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:56.416298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:56.416574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:56.417506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] RBAC: Adding query filter params (role.id=df5e2fcd53214d0ca08b0286b303727c, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=6dad20c291c5440ca6f7c89a3effe871) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:56.417922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:56.421022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131ba53-18f5-4fb4-9384-a3d87b00c9d3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:56.430846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 618/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:01:56 2022] GET /identity/v3/role_assignments?user.id=6dad20c291c5440ca6f7c89a3effe871&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=df5e2fcd53214d0ca08b0286b303727c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:01:58.943451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6ca1-5c3f-4349-b0d3-4af9071546b0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:58.944397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6ca1-5c3f-4349-b0d3-4af9071546b0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:58.944397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6ca1-5c3f-4349-b0d3-4af9071546b0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:58.945443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 629/1247] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:01:58.951957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:58.952823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:58.952823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:58.959092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:58.977459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:01:58.979763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:01:59.089345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 619/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:01:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 17:01:59.138787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b6ca1-5c3f-4349-b0d3-4af9071546b0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:59.178682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:59.334047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b6ca1-5c3f-4349-b0d3-4af9071546b0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:59.396704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:59.403933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:59.403933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:59.403933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:59.419723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:01:59.426629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:59.433239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:59.434979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 630/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:01:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:01:59.437595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:59.592182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57e70d-8ede-4054-ba30-072e3677ff31 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:59.639920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:59.645445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:59.646177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:59.646461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:59.646739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:59.647493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:01:59.647798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:59.649276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:59.660850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 620/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:01:59.667583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:01:59.800653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:01:59.841212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b015a-6009-4b33-add4-f3865a5c7d9a None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:01:59.896340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:01:59.897145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:01:59.897576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:01:59.897994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:01:59.914910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:01:59.917221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:01:59.945970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:01:59.948557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:01:59.951830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:01:59.951830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:01:59.951830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:01:59.951830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:01:59.953274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 631/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:01:59.956598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:00.142902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61c39a-fe4d-446b-8ae9-a749966d8732 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:00.210645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:00.211481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:00.211904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:00.212368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:00.213572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:00.214091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:00.217784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:00.234685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 621/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:00.242622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:00.387922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9a8ed-76e3-4efb-8112-614c870a9435 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:00.438130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:00.454827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:00.455959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:00.456642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:00.457168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:00.471919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:00.474083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 17:02:00.498548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:00.502052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:00.505085 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 632/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:02:00.509557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:00.594629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:00.678782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29552cb3-11b6-4dfb-ada6-e71d01c6456b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:00.736787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:00.738035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:00.738625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:00.739179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:00.740797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:00.741297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:00.743778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:00.770208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 622/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Tue Jul 26 17:02:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:00.778359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:00.966898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b941e-a3a9-4707-bf25-2997eeafd546 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:01.018512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:01.019122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:01.019523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:01.019916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:01.109966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:01.127845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:01.130137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:01.298841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 633/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:02:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:02:01.308304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:01.472407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717ec62-f6da-4021-94c9-877d72346e95 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:01.491969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:01.527130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e23c4-1b97-4414-a457-758469192671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:01.527975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e23c4-1b97-4414-a457-758469192671 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:01.528919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e23c4-1b97-4414-a457-758469192671 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:01.529365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e23c4-1b97-4414-a457-758469192671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:01.547703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e23c4-1b97-4414-a457-758469192671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:01.549735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e23c4-1b97-4414-a457-758469192671 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-031e23c4-1b97-4414-a457-758469192671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:02:01.571290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:01.573485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:01.575597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:02:01.575597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:02:01.575597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:01.575597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:01.575597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 623/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:01.581296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:01.729194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:01.755546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca36236-bf0a-4143-8a8f-7e0360f44a64 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:01.826833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:01.827547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:01.827838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:01.828119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:01.828840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:01.829189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:01.830780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:01.846186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 634/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:01.853872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e23c4-1b97-4414-a457-758469192671 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:02.001454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e23c4-1b97-4414-a457-758469192671 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:02.054138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:02.055117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:02.055543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:02.056194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:02.067385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:02.070015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 17:02:02.092123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:02.094280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:02.095383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 624/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 17:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:02:02.102530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:02.276513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab807e5-2d9b-463f-b0ab-8511af745f58 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:02.331970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:02.332643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:02.332928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:02.333260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:02.334056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:02.334335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:02.336008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338bdeb-c1bf-470a-b671-8df3a5d09c6e None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:02.359819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 635/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 17:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:02.366558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:02.406542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:02.546112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461e56e-654c-4ac2-9b1c-6848b1bb972f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:02.608613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:02.609977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:02.610418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:02.610850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:02.612048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=6dad20c291c5440ca6f7c89a3effe871) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:02.612486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:02.614232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bd6d1-b756-440a-8440-e21a0200c158 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:02.622976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 625/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:02:02 2022] GET /identity/v3/role_assignments?user.id=6dad20c291c5440ca6f7c89a3effe871&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:04.868684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e877f3-23d5-4c88-8709-805da2c0dc64 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:04.869292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e877f3-23d5-4c88-8709-805da2c0dc64 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:04.869292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e877f3-23d5-4c88-8709-805da2c0dc64 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:04.870679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 636/1261] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:02:04.876767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422246f8-f602-49da-90b8-3828222cadcf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:04.877398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422246f8-f602-49da-90b8-3828222cadcf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:04.877398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422246f8-f602-49da-90b8-3828222cadcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:04.897705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-422246f8-f602-49da-90b8-3828222cadcf None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:02:04.898932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-422246f8-f602-49da-90b8-3828222cadcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:02:04.976352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 626/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:02:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:02:05.000620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e877f3-23d5-4c88-8709-805da2c0dc64 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:05.014555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:05.123919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e877f3-23d5-4c88-8709-805da2c0dc64 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:05.177308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:05.178151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:05.178445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:05.178864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:05.194814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:02:05.201018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:05.202878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:05.204523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 637/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:02:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:05.211659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422246f8-f602-49da-90b8-3828222cadcf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:05.234835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:05.362994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422246f8-f602-49da-90b8-3828222cadcf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:05.424201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:05.424850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:05.425164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:05.425437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:05.426182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:05.426485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:05.427910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:05.437393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 627/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:02:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:05.443653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:05.526313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:05.569447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71f890-2436-4cb1-a09c-2b2d45490220 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:05.610118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:05.610846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:05.611172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:05.611495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:05.624065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:05.625679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:02:05.643281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:05.645086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:05.647110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:02:05.647110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:02:05.647110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:05.647110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:05.647110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 638/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:02:05.652300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:05.743002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:05.765880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f465f6-d4ed-4546-b079-1bafd5bce577 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:05.804064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:05.804747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:05.805138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:05.805452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:05.806208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:05.806547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:05.807975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:05.821901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 628/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:05.832448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:06.042051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d185d6-2e11-4d29-89aa-a670a7f02d09 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:06.083463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:06.112691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:06.113870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:06.114377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:06.114867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:06.131007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:06.133883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 17:02:06.154774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:06.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:06.158598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 639/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:06.166282 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:06.309683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465eff5-501a-4df1-abe4-8564d6522cc9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:06.350610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:06.351401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:06.351815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:06.352223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:06.353238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:06.353780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:06.355367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:06.366778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:06.376630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 629/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Tue Jul 26 17:02:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:06.382745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:06.503988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a53645-f2e2-4935-a198-783bbb70ff70 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:06.546046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:06.546743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:06.547169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:06.547450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:06.561600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:06.563113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:02:06.579956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:06.581734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:06.583554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:02:06.583554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:02:06.583554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:06.583554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:06.583554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 640/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:06.588340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:06.706104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34320955-8044-44cb-a48c-c03399b1ffec None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:06.751400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:06.752066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:06.752351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:06.752628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:06.753473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:06.753814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:06.755356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:06.770135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 630/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:02:06.776550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:06.815184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:06.902997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d66d4-3ee8-402f-ba40-95989deb9800 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:06.944930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:06.945614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:06.945900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:06.946181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:06.954962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:06.956458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 17:02:06.970052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:06.971825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:06.973651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 641/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 17:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:06.978873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:07.049984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:07.126851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536b78f-cdac-4348-9d0d-261e8ef563bc None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:07.180715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:07.181822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:07.182261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:07.182721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:07.183619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:07.184583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:07.186216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd2304-66c5-4a1d-8292-e6aadb7ec2b5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:07.200950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 631/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 17:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:07.207358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:07.323532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:07.341134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878c80c-b847-4b3a-aa5d-867846076344 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:07.382144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacee3b-2871-4d25-9792-0842fe2149b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:07.382822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacee3b-2871-4d25-9792-0842fe2149b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:07.383492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacee3b-2871-4d25-9792-0842fe2149b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:07.383841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacee3b-2871-4d25-9792-0842fe2149b0 None admin] PATH_INFO: `/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/6dad20c291c5440ca6f7c89a3effe871/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:07.386149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacee3b-2871-4d25-9792-0842fe2149b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c34156723c24f918ca8fe84be2c7774, user_id=6dad20c291c5440ca6f7c89a3effe871, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:07.388289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacee3b-2871-4d25-9792-0842fe2149b0 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:07.406670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 642/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Tue Jul 26 17:02:07 2022] PUT /identity/v3/projects/8c34156723c24f918ca8fe84be2c7774/users/6dad20c291c5440ca6f7c89a3effe871/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:02:09.503879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ce80e-3eee-443e-9647-e16618942664 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:09.504440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ce80e-3eee-443e-9647-e16618942664 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:09.504440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ce80e-3eee-443e-9647-e16618942664 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:09.505963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 632/1274] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:02:09.511969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90335e0f-af7d-464b-83c7-567358053235 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:09.512557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90335e0f-af7d-464b-83c7-567358053235 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:09.513087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90335e0f-af7d-464b-83c7-567358053235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:09.534123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-90335e0f-af7d-464b-83c7-567358053235 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:02:09.535415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-90335e0f-af7d-464b-83c7-567358053235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:02:09.627877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 643/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:02:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:02:09.653500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ce80e-3eee-443e-9647-e16618942664 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:09.781620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ce80e-3eee-443e-9647-e16618942664 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:09.803135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:09.824018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:09.825688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:09.825971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:09.826243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:09.838860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 17:02:09.845313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:09.846537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:09.847789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 633/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Tue Jul 26 17:02:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:02:09.856452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90335e0f-af7d-464b-83c7-567358053235 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:09.951876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:09.973198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90335e0f-af7d-464b-83c7-567358053235 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:10.011705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:10.012590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:10.012867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:10.013170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:10.013912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] RBAC: Adding query filter params (name=admin) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:10.014215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:10.015632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:10.025648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 644/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Tue Jul 26 17:02:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:02:10.032582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:10.153678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b81e9-e920-4e91-8b93-de7a496c954b None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:10.206286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:10.207188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:10.207615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:10.208052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:10.225064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:10.226659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:02:10.245609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:10.247333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:10.253245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:02:10.253245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:02:10.253245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:10.253245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:10.253245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 634/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:10.253926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:10.374680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28d4fa-fa93-43a7-8b1f-78ad2cd13ede None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:10.414092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:10.414791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:10.415105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:10.415411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:10.416153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:10.416518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:10.418595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:10.432907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 645/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:10.439673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:10.477806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:10.561425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c8b9e-b3fe-422f-84b2-680d8f5a6f5f None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:10.601172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:10.601857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:10.602135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:10.602407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:10.612331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:10.614544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 17:02:10.627497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:10.629291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:10.631610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 635/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:10.635359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:10.639600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:10.826127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ced99b-e971-4325-aac3-7696a5633fb6 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:10.879265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:10.879902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:10.880182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:10.880454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:10.881290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:10.881596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:10.883041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:10.903202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 646/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Tue Jul 26 17:02:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:10.909401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:11.021715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:11.033737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d94228-db0d-4561-ac42-bdbc4cfe0df9 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:11.076246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:11.076805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:11.077111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:11.077385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:11.171863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:11.173396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:11.281937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:11.294057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 636/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Tue Jul 26 17:02:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 17:02:11.300519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:11.373908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:11.430756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efa3e1-bdc6-4985-86ba-6d08caeb75ca None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:11.469676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:11.470395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:11.470719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:11.471035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:11.482841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:11.484668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 17:02:11.502026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:11.503268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:11.504379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 17:02:11.504379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 17:02:11.504379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 17:02:11.504379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:11.504627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 647/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 17:02:11.514200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:11.640700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169169cb-f605-4b5b-96d8-d76f8dec43b5 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:11.683710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:11.684355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:11.684694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:11.684901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:11.685704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:11.686011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:11.687435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:11.701456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 637/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Tue Jul 26 17:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:11.707626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:11.827487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c741b-3717-48dc-ac67-0bda2b99798f None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:11.846711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:11.882940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:11.883977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:11.884425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:11.884863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:11.897462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:11.900155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 17:02:11.924037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:11.925818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: ERROR keystone.server.flask.application Jul 26 17:02:11.927535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 648/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Tue Jul 26 17:02:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:11.936277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:11.996368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:12.057653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d6184-3b1f-49df-ab1f-1b15c3f75c8c None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:12.098026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:12.098659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:12.098936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:12.099204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:12.099977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:12.100279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:12.101795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01ed34-a9f0-4595-8fb7-77a2794335e3 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:12.116746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 638/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Tue Jul 26 17:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:12.123438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:12.297947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffb955-a872-43ed-b21b-ebf18c2e9988 None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:12.338121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:12.338750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:12.339030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:12.339298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:12.340685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] RBAC: Adding query filter params (role.id=a05783d7bc51415a97b5a2545cdd7cc9, scope.project.id=8c34156723c24f918ca8fe84be2c7774, user.id=6dad20c291c5440ca6f7c89a3effe871) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:12.341017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:12.342682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2cba4-deb8-416c-b4fe-5d1fb529d978 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:12.349500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 649/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Tue Jul 26 17:02:12 2022] GET /identity/v3/role_assignments?user.id=6dad20c291c5440ca6f7c89a3effe871&scope.project.id=8c34156723c24f918ca8fe84be2c7774&role.id=a05783d7bc51415a97b5a2545cdd7cc9 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:02:14.534751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80d78f-4cf8-4ffe-baa8-be569a92c486 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:14.535334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80d78f-4cf8-4ffe-baa8-be569a92c486 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:14.535334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80d78f-4cf8-4ffe-baa8-be569a92c486 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:14.536861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 639/1288] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:02:14.543495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:14.544064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:14.544064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:14.568246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:02:14.569526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:02:14.618940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:14.680453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 650/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:02:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:02:14.706313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80d78f-4cf8-4ffe-baa8-be569a92c486 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:14.880393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80d78f-4cf8-4ffe-baa8-be569a92c486 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:14.898348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:14.933285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:14.934036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:14.934547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:14.934859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:14.935515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:14.936946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:14.951276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:14.953292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:14.955009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 640/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 17:02:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:14.961627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:15.111789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7e156-79c7-48b0-8214-7ff9e4ae9258 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:15.166053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:15.167184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:15.167694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:15.168735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:15.170155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:15.170695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:15.173570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29287c4-5b67-48ad-b2d2-d9bfa51257b3 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:15.180713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 651/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Tue Jul 26 17:02:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:15.188639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:15.312728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cc83b-517a-4fd9-8a73-be225153434a None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:15.361445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:15.362098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:15.362377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:15.362652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] PATH_INFO: `/v3/services` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:15.363360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:15.363666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:15.365127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2413c6-9884-427f-9948-713f28e44a74 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:15.370332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 641/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Tue Jul 26 17:02:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:17.517843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ceca0-3b02-4dde-a549-40115bb7bdab None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:17.518448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ceca0-3b02-4dde-a549-40115bb7bdab None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:17.518448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ceca0-3b02-4dde-a549-40115bb7bdab None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:17.519849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 652/1293] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:02:17.525900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ccda5-8961-4ae5-aeb9-91ddb4fb57c0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:17.526746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ccda5-8961-4ae5-aeb9-91ddb4fb57c0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:17.526746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ccda5-8961-4ae5-aeb9-91ddb4fb57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:17.547532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1f8ccda5-8961-4ae5-aeb9-91ddb4fb57c0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:02:17.548774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1f8ccda5-8961-4ae5-aeb9-91ddb4fb57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:02:17.583220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:17.627900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 642/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:02:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 17:02:17.651446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ceca0-3b02-4dde-a549-40115bb7bdab None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:17.687566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:17.775361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ceca0-3b02-4dde-a549-40115bb7bdab None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:17.815567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a4a32-5840-4e5b-99fd-474dbf7cce37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:17.816265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a4a32-5840-4e5b-99fd-474dbf7cce37 None admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:17.816601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a4a32-5840-4e5b-99fd-474dbf7cce37 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:17.816919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a4a32-5840-4e5b-99fd-474dbf7cce37 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:17.817718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a4a32-5840-4e5b-99fd-474dbf7cce37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:17.819192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a4a32-5840-4e5b-99fd-474dbf7cce37 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:17.835813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 653/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Tue Jul 26 17:02:17 2022] POST /identity/v3/services => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:02:20.003254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0404-1521-45cd-ac65-507ab322c092 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:20.003826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0404-1521-45cd-ac65-507ab322c092 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:20.003826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0404-1521-45cd-ac65-507ab322c092 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:20.005257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 643/1296] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:02:20.013398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:20.014003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:20.014003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:20.033710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:02:20.034937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:02:20.124997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 654/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:02:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:02:20.165578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0404-1521-45cd-ac65-507ab322c092 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:20.356425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0404-1521-45cd-ac65-507ab322c092 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:20.419369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:20.420394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:20.420809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:20.421339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:20.422515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:20.425073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:20.442776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:20.444056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:20.445153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 644/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 17:02:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:20.454055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:20.474447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:20.568150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc56f9a-6d8c-4603-a02e-2939a0ea7f21 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:20.609661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:20.610244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:20.610523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:20.610800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:20.611496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:20.611799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:20.613261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:20.619508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 655/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Tue Jul 26 17:02:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:20.625966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:20.671125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:20.798881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712661ed-3a4f-4d9c-9c27-f011d1d7d347 None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:20.862208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:20.863142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:20.863642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:20.864132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:20.865498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] RBAC: Adding query filter params (interface=public, service_id=48c6f26138634c7aa879e7fea33abbf7, region_id=RegionOne) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:20.866065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:20.868610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cf6e36-1281-4534-8915-9032a4659e37 None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:20.878397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 645/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Tue Jul 26 17:02:20 2022] GET /identity/v3/endpoints?service_id=48c6f26138634c7aa879e7fea33abbf7&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:02:20.884202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:20.983534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:21.014744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06f6fd-fb80-4c8f-9985-283df3c1079c None admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:21.067662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a0dd2-d142-4b81-83b3-d4b063e25cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:21.068722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0dd2-d142-4b81-83b3-d4b063e25cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:21.069173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0dd2-d142-4b81-83b3-d4b063e25cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:21.069621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a0dd2-d142-4b81-83b3-d4b063e25cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:21.070781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a0dd2-d142-4b81-83b3-d4b063e25cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:21.073270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a0dd2-d142-4b81-83b3-d4b063e25cc7 None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:21.093051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 656/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Tue Jul 26 17:02:20 2022] GET /identity/v3/services => generated 2191 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:02:23.185973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a0b1c-cf4e-4110-81bb-760a2fb708c9 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:23.186786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a0b1c-cf4e-4110-81bb-760a2fb708c9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:23.186943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a0b1c-cf4e-4110-81bb-760a2fb708c9 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:23.188358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 646/1302] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:02:23.201922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:23.201922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:23.201922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:23.216347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:02:23.217666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:02:23.294992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 657/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:02:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 17:02:23.317471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a0b1c-cf4e-4110-81bb-760a2fb708c9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:23.431996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:23.440845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a0b1c-cf4e-4110-81bb-760a2fb708c9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:23.481511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:23.482169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:23.482447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:23.482717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:23.483357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:23.485055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 17:02:23.494764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 17:02:23.496702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application with Lock( Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self._enter() Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application return self.creator() Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application created_value = creator( Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 17:02:23.498502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: ERROR keystone.server.flask.application Jul 26 17:02:23.501236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 647/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Tue Jul 26 17:02:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 17:02:23.502591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:23.594462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:23.622285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7345f4-d0d1-436f-97e0-1f2acb2db0c8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:23.663198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:23.663887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:23.664211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:23.664531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] PATH_INFO: `/v3/services` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:23.665332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:02:23.665680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:23.667130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da57f9-5167-4cd8-b581-bac4bf29afeb None admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:23.673599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 658/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Tue Jul 26 17:02:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:02:23.679777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:02:23.800105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912049f-1899-4cfc-82c6-0b2886994dfd None admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:02:23.839392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb88168-fe81-40a8-ab03-015742c4dbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:02:23.840034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb88168-fe81-40a8-ab03-015742c4dbfa None admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:02:23.840312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb88168-fe81-40a8-ab03-015742c4dbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:02:23.840583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb88168-fe81-40a8-ab03-015742c4dbfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:02:23.841380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb88168-fe81-40a8-ab03-015742c4dbfa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:02:23.842799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb88168-fe81-40a8-ab03-015742c4dbfa None admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:02:23.866405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 648/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:02:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:02:33.670266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=30 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:02:33.861189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:39.514725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654e3b-7772-49fe-ae86-80045b89f1f4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:39.516333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654e3b-7772-49fe-ae86-80045b89f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:39.517203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654e3b-7772-49fe-ae86-80045b89f1f4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:39.520332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 659/1307] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:03:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:03:39.529744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de26418-fc4d-4f2f-9f64-9d8afdb22f04 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:39.530605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de26418-fc4d-4f2f-9f64-9d8afdb22f04 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:39.530750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de26418-fc4d-4f2f-9f64-9d8afdb22f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:39.741581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5de26418-fc4d-4f2f-9f64-9d8afdb22f04 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:39.744522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5de26418-fc4d-4f2f-9f64-9d8afdb22f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:40.059895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 649/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 532 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:03:40.162759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735348-3a2c-42c5-94f9-2ddf47035298 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:40.163511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735348-3a2c-42c5-94f9-2ddf47035298 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:40.164126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735348-3a2c-42c5-94f9-2ddf47035298 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:40.169229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 660/1309] 10.222.0.48 () {54 vars in 965 bytes} [Tue Jul 26 17:03:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:03:40.179448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04146c8-da96-478d-8ecc-05a22d0f9ae7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:40.180186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04146c8-da96-478d-8ecc-05a22d0f9ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:40.181053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04146c8-da96-478d-8ecc-05a22d0f9ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:40.454846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f04146c8-da96-478d-8ecc-05a22d0f9ae7 None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:40.457463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f04146c8-da96-478d-8ecc-05a22d0f9ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:40.539412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:40.670026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 650/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Tue Jul 26 17:03:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:03:40.680465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735348-3a2c-42c5-94f9-2ddf47035298 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:03:40.685276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-68735348-3a2c-42c5-94f9-2ddf47035298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:40.847577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735348-3a2c-42c5-94f9-2ddf47035298 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:03:40.959472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:03:40.960110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:40.960390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:40.960648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:40.963488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:41.043983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:41.129312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:03:41.133088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdeadc5-77f4-4762-846e-13fabd74a723 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:03:41.176430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 661/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:03:44.115508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507f0e5-f925-4a0e-9ed5-a56f36d6e7cc None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:44.116116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507f0e5-f925-4a0e-9ed5-a56f36d6e7cc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:44.116116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507f0e5-f925-4a0e-9ed5-a56f36d6e7cc None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:44.117473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 651/1312] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:03:44.125066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537590-73cb-4c00-8d93-464315a41d11 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:44.125598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537590-73cb-4c00-8d93-464315a41d11 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:44.125713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537590-73cb-4c00-8d93-464315a41d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:44.210134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-61537590-73cb-4c00-8d93-464315a41d11 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:44.211461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-61537590-73cb-4c00-8d93-464315a41d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:44.326225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 662/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:03:44.386384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24d95-be53-4b91-98be-833e247b8b7f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:44.386632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24d95-be53-4b91-98be-833e247b8b7f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:44.386946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b24d95-be53-4b91-98be-833e247b8b7f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:44.388482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 652/1314] 10.222.0.48 () {54 vars in 965 bytes} [Tue Jul 26 17:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:03:44.395455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cef68-4a17-4d65-b874-5b711c344ec6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:44.395790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cef68-4a17-4d65-b874-5b711c344ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:44.396107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cef68-4a17-4d65-b874-5b711c344ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:44.411142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:44.498047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a66cef68-4a17-4d65-b874-5b711c344ec6 None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:44.499375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a66cef68-4a17-4d65-b874-5b711c344ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:44.590438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 663/1315] 10.222.0.48 () {58 vars in 1062 bytes} [Tue Jul 26 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:03:44.600537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b24d95-be53-4b91-98be-833e247b8b7f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:03:44.602505 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-84b24d95-be53-4b91-98be-833e247b8b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:44.726783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b24d95-be53-4b91-98be-833e247b8b7f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:03:44.818397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:44.821263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:03:44.821924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:44.822551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:44.823108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:44.825238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:44.953638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:03:44.955118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30ecac-4f02-495b-91d1-3672e5d7ab6f service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:03:44.986314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 653/1316] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:03:47.688455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936090e4-83c4-463d-8600-98e662c51958 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:47.689099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936090e4-83c4-463d-8600-98e662c51958 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:47.689099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936090e4-83c4-463d-8600-98e662c51958 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:47.690641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 664/1317] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:03:47.697410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023eb2-7b45-42c6-b17c-1f065d6b66dd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:47.698008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023eb2-7b45-42c6-b17c-1f065d6b66dd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:47.698008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023eb2-7b45-42c6-b17c-1f065d6b66dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:47.778561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e0023eb2-7b45-42c6-b17c-1f065d6b66dd None None] MFA Rules not processed for user `a4e2d0e5269b4930bf2676c185ddad57`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:47.779834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e0023eb2-7b45-42c6-b17c-1f065d6b66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:47.849359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:47.907695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 654/1318] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:03:47.953943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936090e4-83c4-463d-8600-98e662c51958 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:03:47.955781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-936090e4-83c4-463d-8600-98e662c51958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:48.093168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936090e4-83c4-463d-8600-98e662c51958 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:03:48.096534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:48.195131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:03:48.195680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:48.195956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:48.196814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:48.198996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:48.345815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:03:48.347294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ff951-326e-46ad-9b63-e13e2a7b6002 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:03:48.369351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 665/1319] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:03:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:03:51.292674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91b62-0706-49a4-a67e-b212f5a8409a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:51.293304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91b62-0706-49a4-a67e-b212f5a8409a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:51.293304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91b62-0706-49a4-a67e-b212f5a8409a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:51.294687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 655/1320] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:03:51.300721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2a834-3dad-4835-86d6-9276c2afb729 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:51.301355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2a834-3dad-4835-86d6-9276c2afb729 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:51.301506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2a834-3dad-4835-86d6-9276c2afb729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:51.359073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:51.393905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5bd2a834-3dad-4835-86d6-9276c2afb729 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:51.395172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5bd2a834-3dad-4835-86d6-9276c2afb729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:51.564994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 666/1321] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:03:51.606570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91b62-0706-49a4-a67e-b212f5a8409a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:03:51.608188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bfb91b62-0706-49a4-a67e-b212f5a8409a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:51.775877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91b62-0706-49a4-a67e-b212f5a8409a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:03:51.897754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:03:51.898827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:51.899819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:51.900310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:51.903827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:51.918936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:52.134329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:03:52.137310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc413de4-159c-4d15-a2e3-94838a2b271b service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:03:52.166574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 656/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:03:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:03:55.272115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40636eb2-2e07-4756-9c5e-485d587d885d None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:55.272766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40636eb2-2e07-4756-9c5e-485d587d885d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:55.272766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40636eb2-2e07-4756-9c5e-485d587d885d None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:55.274962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 667/1323] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:03:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:03:55.282879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f1486-2937-4410-aa5a-7f769b25f28c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:55.283697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f1486-2937-4410-aa5a-7f769b25f28c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:55.283906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f1486-2937-4410-aa5a-7f769b25f28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:55.396223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-682f1486-2937-4410-aa5a-7f769b25f28c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:55.398350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-682f1486-2937-4410-aa5a-7f769b25f28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:55.482822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:55.562597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 657/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:03:55.603599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40636eb2-2e07-4756-9c5e-485d587d885d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:03:55.605456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-40636eb2-2e07-4756-9c5e-485d587d885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:55.762984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40636eb2-2e07-4756-9c5e-485d587d885d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:03:55.778203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:55.869331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:03:55.870451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:55.871214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:55.871615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:55.873857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:56.041634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:03:56.043326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599eb90-5b93-4ea1-8840-5c7d6d6576b9 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:03:56.072731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 668/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:03:58.330673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325985f-2737-4cf6-a185-461024565b0b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:58.331349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325985f-2737-4cf6-a185-461024565b0b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:58.331609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325985f-2737-4cf6-a185-461024565b0b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:58.334465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 658/1326] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:03:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:03:58.342659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480508b-2373-40f1-9261-0344da069843 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:58.343488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480508b-2373-40f1-9261-0344da069843 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:58.343751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480508b-2373-40f1-9261-0344da069843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:58.427510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:58.453654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b480508b-2373-40f1-9261-0344da069843 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:03:58.455172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b480508b-2373-40f1-9261-0344da069843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:58.642418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 669/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:03:58.698956 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325985f-2737-4cf6-a185-461024565b0b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:03:58.701725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5325985f-2737-4cf6-a185-461024565b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:58.869995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325985f-2737-4cf6-a185-461024565b0b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:03:58.971529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:03:58.972191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:03:58.972586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:03:58.972925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:03:58.974804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:03:58.995671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:03:59.119245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:03:59.121665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bcac4-2fa4-4e39-9877-a431cbc6f989 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:03:59.153222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 659/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:01.732364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9102c-b700-4662-aaee-0130cb6dc19f None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:01.733081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9102c-b700-4662-aaee-0130cb6dc19f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:01.733252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9102c-b700-4662-aaee-0130cb6dc19f None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:01.736077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 670/1329] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:01.742460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5833b-757e-4a51-86d3-d8d1e1eac98c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:01.743936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5833b-757e-4a51-86d3-d8d1e1eac98c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:01.743936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5833b-757e-4a51-86d3-d8d1e1eac98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:01.830139 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-86e5833b-757e-4a51-86d3-d8d1e1eac98c None None] MFA Rules not processed for user `a4e2d0e5269b4930bf2676c185ddad57`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:01.831535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-86e5833b-757e-4a51-86d3-d8d1e1eac98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:01.913159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:01.956270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 660/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:04:02.003162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9102c-b700-4662-aaee-0130cb6dc19f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:02.004870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5ca9102c-b700-4662-aaee-0130cb6dc19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:02.152216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9102c-b700-4662-aaee-0130cb6dc19f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:02.191146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:02.307093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:02.308068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:02.308505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:02.308931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:02.311777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:02.558514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:02.561130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3febb-d25e-48b6-877d-5484b5952356 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:02.599153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 671/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:04:05.291748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaca09-ec4e-4f89-9771-80a672dc7460 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:05.292419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaca09-ec4e-4f89-9771-80a672dc7460 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:05.292419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaca09-ec4e-4f89-9771-80a672dc7460 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:05.293655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 661/1332] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:05.299620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1524402-8517-45b9-a595-7b1741bd719d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:05.300211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1524402-8517-45b9-a595-7b1741bd719d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:05.300363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1524402-8517-45b9-a595-7b1741bd719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:05.378199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:05.382011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e1524402-8517-45b9-a595-7b1741bd719d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:05.384113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e1524402-8517-45b9-a595-7b1741bd719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:05.567335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 672/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:05.606480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaca09-ec4e-4f89-9771-80a672dc7460 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:05.608308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7ebaca09-ec4e-4f89-9771-80a672dc7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:05.746959 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaca09-ec4e-4f89-9771-80a672dc7460 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:05.878271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:05.879724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:05.880135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:05.880755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:05.883313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:05.940507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:06.073908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:06.075185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346ee7a-e6b5-4a13-be13-08b8c2f99b6c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:06.098950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 662/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:08.354736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bcdb3-9f02-410c-818f-d4aca66afe2a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:08.355336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bcdb3-9f02-410c-818f-d4aca66afe2a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:08.355336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bcdb3-9f02-410c-818f-d4aca66afe2a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:08.356748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 673/1335] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:08.363486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3e9fb-b22d-4013-8896-cd11b791728f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:08.364171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3e9fb-b22d-4013-8896-cd11b791728f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:08.364171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3e9fb-b22d-4013-8896-cd11b791728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:08.452471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-69a3e9fb-b22d-4013-8896-cd11b791728f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:08.454412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-69a3e9fb-b22d-4013-8896-cd11b791728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:08.556100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:08.584440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 663/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:08.650426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bcdb3-9f02-410c-818f-d4aca66afe2a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:08.651992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fe8bcdb3-9f02-410c-818f-d4aca66afe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:08.846006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bcdb3-9f02-410c-818f-d4aca66afe2a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:08.900505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:08.988016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:08.989117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:08.989629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:08.990122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:08.994049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:09.192390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:09.193820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa427aef-2a74-4dea-80d7-e3caf71405d8 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:09.217981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 674/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:04:11.867054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8132e-189b-4b86-806a-4f090808fa9c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:11.868204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8132e-189b-4b86-806a-4f090808fa9c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:11.868335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8132e-189b-4b86-806a-4f090808fa9c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:11.869866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 664/1338] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:11.876698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b971-7c9a-46e0-833d-879dcd66681e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:11.877264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b971-7c9a-46e0-833d-879dcd66681e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:11.877717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b971-7c9a-46e0-833d-879dcd66681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:11.978771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f2a1b971-7c9a-46e0-833d-879dcd66681e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:11.980030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f2a1b971-7c9a-46e0-833d-879dcd66681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:11.990491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:12.130204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 675/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:12.173753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8132e-189b-4b86-806a-4f090808fa9c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:12.175374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7ca8132e-189b-4b86-806a-4f090808fa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:12.296414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8132e-189b-4b86-806a-4f090808fa9c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:12.388638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:12.389614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:12.390099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:12.390562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:12.393802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:12.467020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:12.576123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:12.577666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c577e50-01ca-4996-9841-ba484b719e82 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:12.601227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 665/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:17.834624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5caef-7570-4f6e-8a5c-ade2c05cae16 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:17.835442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5caef-7570-4f6e-8a5c-ade2c05cae16 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:17.835442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5caef-7570-4f6e-8a5c-ade2c05cae16 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:17.837781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 676/1341] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:17.844673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5b32-450e-4b3c-96d6-e97f84ff81b9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:17.845540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5b32-450e-4b3c-96d6-e97f84ff81b9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:17.845869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a5b32-450e-4b3c-96d6-e97f84ff81b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:17.966672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c17a5b32-450e-4b3c-96d6-e97f84ff81b9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:17.967969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c17a5b32-450e-4b3c-96d6-e97f84ff81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:18.111035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:18.128692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 666/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:18.192950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5caef-7570-4f6e-8a5c-ade2c05cae16 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:18.196336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7dc5caef-7570-4f6e-8a5c-ade2c05cae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:18.331023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5caef-7570-4f6e-8a5c-ade2c05cae16 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:18.403037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:18.471495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:18.472949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:18.473626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:18.474224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:18.478652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:18.645277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:18.646710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f88392-eafa-4730-bbe0-eccbe5760ed4 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:18.669914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 677/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:04:21.977347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394f11a-12d3-4c94-a947-7acc1c26825c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:21.978368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394f11a-12d3-4c94-a947-7acc1c26825c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:21.978368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394f11a-12d3-4c94-a947-7acc1c26825c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:21.980193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 667/1344] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:21.987544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cf2da-eb35-4d48-a213-5eecfc9241af None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:21.988379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cf2da-eb35-4d48-a213-5eecfc9241af None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:21.988522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cf2da-eb35-4d48-a213-5eecfc9241af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:22.084904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-469cf2da-eb35-4d48-a213-5eecfc9241af None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:22.087616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-469cf2da-eb35-4d48-a213-5eecfc9241af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:22.120875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:22.293260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 678/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:04:22.358390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394f11a-12d3-4c94-a947-7acc1c26825c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:22.360446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0394f11a-12d3-4c94-a947-7acc1c26825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:22.504709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394f11a-12d3-4c94-a947-7acc1c26825c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:22.608832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:22.609844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:22.610246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:22.610626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:22.612708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:22.672468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:22.759488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:22.761008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda2ae19-43ef-46fb-82a8-bebb28e28619 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:22.785655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 668/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:28.064831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c70bb-9c3e-4735-8818-642c5c20e481 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:28.065648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c70bb-9c3e-4735-8818-642c5c20e481 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:28.065648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c70bb-9c3e-4735-8818-642c5c20e481 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:28.066833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 679/1347] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:28.073381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9595139-9801-485f-bd5e-43eb132c4aef None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:28.074156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9595139-9801-485f-bd5e-43eb132c4aef None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:28.074156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9595139-9801-485f-bd5e-43eb132c4aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:28.169644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a9595139-9801-485f-bd5e-43eb132c4aef None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:28.171568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a9595139-9801-485f-bd5e-43eb132c4aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:28.325866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:28.333515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 669/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:28.379734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c70bb-9c3e-4735-8818-642c5c20e481 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:28.381375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-153c70bb-9c3e-4735-8818-642c5c20e481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:28.542913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c70bb-9c3e-4735-8818-642c5c20e481 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:28.614618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:28.646948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:28.647588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:28.647872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:28.648688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:28.650502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:28.837904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:28.840769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb9165-d42d-4cb0-a608-e2407e823d06 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:28.870915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 680/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:34.925009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d54d-ef35-4dd7-8d44-df9fb8f2c104 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:34.925894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d54d-ef35-4dd7-8d44-df9fb8f2c104 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:34.925894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d54d-ef35-4dd7-8d44-df9fb8f2c104 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:34.928476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 670/1350] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:04:34.937123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d999a-8168-45bb-bfb8-bb6d66a2acf0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:34.937686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d999a-8168-45bb-bfb8-bb6d66a2acf0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:34.938319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d999a-8168-45bb-bfb8-bb6d66a2acf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:35.084545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-244d999a-8168-45bb-bfb8-bb6d66a2acf0 None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:35.086736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-244d999a-8168-45bb-bfb8-bb6d66a2acf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:35.151834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:35.342960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 681/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:04:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:40.442456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3ae4-f14b-434a-94f9-4533979deeec None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:40.445680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3ae4-f14b-434a-94f9-4533979deeec None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:40.446581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c3ae4-f14b-434a-94f9-4533979deeec None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:40.449720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 671/1352] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:04:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:40.456505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b09d7-91d6-4f05-9a96-75586c671e09 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:40.458761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b09d7-91d6-4f05-9a96-75586c671e09 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:40.459389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b09d7-91d6-4f05-9a96-75586c671e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:40.601896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-211b09d7-91d6-4f05-9a96-75586c671e09 None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:40.603367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-211b09d7-91d6-4f05-9a96-75586c671e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:40.705192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:40.760875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 682/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:04:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:42.379251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0058e-0c79-4edc-a4e5-0a7fb1ca920e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:42.379797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0058e-0c79-4edc-a4e5-0a7fb1ca920e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:42.379880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0058e-0c79-4edc-a4e5-0a7fb1ca920e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:42.381450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 672/1354] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:04:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:42.388427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364677f-2db1-4e8f-bdd4-03c169b34adc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:42.389061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364677f-2db1-4e8f-bdd4-03c169b34adc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:42.389061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364677f-2db1-4e8f-bdd4-03c169b34adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:42.487577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5364677f-2db1-4e8f-bdd4-03c169b34adc None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:42.488807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5364677f-2db1-4e8f-bdd4-03c169b34adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:42.630011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 683/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:04:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:46.310639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57289d-7400-469b-9f90-832b629d6b7b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:46.311321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57289d-7400-469b-9f90-832b629d6b7b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:46.311590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57289d-7400-469b-9f90-832b629d6b7b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:46.314090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 673/1356] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:46.323218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ac6e7-dc5a-4966-bdba-6c94c3fcb106 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:46.324934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ac6e7-dc5a-4966-bdba-6c94c3fcb106 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:46.325707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ac6e7-dc5a-4966-bdba-6c94c3fcb106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:46.402306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:46.489635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ec5ac6e7-dc5a-4966-bdba-6c94c3fcb106 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:46.494370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ec5ac6e7-dc5a-4966-bdba-6c94c3fcb106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:46.669302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 684/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:46.767107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b46ae8-4c45-44b8-808e-9c1f832840de None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:46.770371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b46ae8-4c45-44b8-808e-9c1f832840de None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:46.771366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b46ae8-4c45-44b8-808e-9c1f832840de None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:46.775483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 674/1358] 10.222.0.48 () {54 vars in 962 bytes} [Tue Jul 26 17:04:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:46.793615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4b13e-b528-4f78-b9e4-0d737d02f9f3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:46.793904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4b13e-b528-4f78-b9e4-0d737d02f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:46.794217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f4b13e-b528-4f78-b9e4-0d737d02f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:46.942903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-99f4b13e-b528-4f78-b9e4-0d737d02f9f3 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:46.945584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-99f4b13e-b528-4f78-b9e4-0d737d02f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:47.045228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:47.186097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 685/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Tue Jul 26 17:04:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:47.194343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b46ae8-4c45-44b8-808e-9c1f832840de None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:47.196066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e6b46ae8-4c45-44b8-808e-9c1f832840de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:47.384089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b46ae8-4c45-44b8-808e-9c1f832840de None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:47.482596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:47.534643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:47.535470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:47.535902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:47.536316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:47.538810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:47.666988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:47.669899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f6222-8d03-400d-9961-e842298d3f9d service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:47.697441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 675/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:50.176935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d2e6c-7cc3-4227-8b0a-349b80648f2e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:50.177538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d2e6c-7cc3-4227-8b0a-349b80648f2e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:50.177538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d2e6c-7cc3-4227-8b0a-349b80648f2e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:50.179554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 686/1361] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:50.185028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef12748-09f7-4c98-8b18-d37831d82b4e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:50.185555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef12748-09f7-4c98-8b18-d37831d82b4e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:50.185672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef12748-09f7-4c98-8b18-d37831d82b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:50.263575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9ef12748-09f7-4c98-8b18-d37831d82b4e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:50.265167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9ef12748-09f7-4c98-8b18-d37831d82b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:50.312561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:50.409648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 676/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:50.489606 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d2e6c-7cc3-4227-8b0a-349b80648f2e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:50.492807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c98d2e6c-7cc3-4227-8b0a-349b80648f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:50.846164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d2e6c-7cc3-4227-8b0a-349b80648f2e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:50.877482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:51.073930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:51.074978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:51.075540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:51.075540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:51.078860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:51.221057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:51.225320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f96e93-6dd1-4d18-b10c-0e03b3c4ad0f service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:51.254050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 687/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:54.458719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82492a96-2a23-4ae6-af13-4c6830b384eb None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:54.459292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82492a96-2a23-4ae6-af13-4c6830b384eb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:54.459732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82492a96-2a23-4ae6-af13-4c6830b384eb None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:54.462605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 677/1364] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:54.468705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f78cb4-9c7c-4281-ab16-3883c0a4c8ba None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:54.469553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f78cb4-9c7c-4281-ab16-3883c0a4c8ba None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:54.469553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f78cb4-9c7c-4281-ab16-3883c0a4c8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:54.489072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:54.583584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-13f78cb4-9c7c-4281-ab16-3883c0a4c8ba None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:54.584901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-13f78cb4-9c7c-4281-ab16-3883c0a4c8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:54.707594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 688/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:04:54.763569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f4d0-c0d3-446b-b1c3-b1f7f5b3b19b None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:54.764082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f4d0-c0d3-446b-b1c3-b1f7f5b3b19b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:54.764437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0f4d0-c0d3-446b-b1c3-b1f7f5b3b19b None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:54.766595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 678/1366] 10.222.0.48 () {54 vars in 962 bytes} [Tue Jul 26 17:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:04:54.775059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998882b-a572-4c3d-b441-7c7b4c3bcb3a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:54.775643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998882b-a572-4c3d-b441-7c7b4c3bcb3a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:54.776017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998882b-a572-4c3d-b441-7c7b4c3bcb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:54.885333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2998882b-a572-4c3d-b441-7c7b4c3bcb3a None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:54.886839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2998882b-a572-4c3d-b441-7c7b4c3bcb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:54.939181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:55.072895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 689/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Tue Jul 26 17:04:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:55.080998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f4d0-c0d3-446b-b1c3-b1f7f5b3b19b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:55.082548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f0e0f4d0-c0d3-446b-b1c3-b1f7f5b3b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:55.285052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0f4d0-c0d3-446b-b1c3-b1f7f5b3b19b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:55.316424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:55.498232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:55.499267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:55.499267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:55.499509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:55.501492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:55.623992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:55.628231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c78e3f-80f0-447b-83a9-e6d71a9a1e8f service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:55.667924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 679/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:04:56.104364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565709d-29fd-4c54-9a97-b5460c1f1d96 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:56.105018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565709d-29fd-4c54-9a97-b5460c1f1d96 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:56.105018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565709d-29fd-4c54-9a97-b5460c1f1d96 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:56.106593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 690/1369] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:56.112582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe387a10-0ac1-453a-a86f-89f442161205 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:56.112869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe387a10-0ac1-453a-a86f-89f442161205 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:56.113197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe387a10-0ac1-453a-a86f-89f442161205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:56.151317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:56.196750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fe387a10-0ac1-453a-a86f-89f442161205 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:56.198057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fe387a10-0ac1-453a-a86f-89f442161205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:56.318773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 680/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:56.655615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200cbdf-b708-44e0-9349-39b23b8169bf None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:56.656433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200cbdf-b708-44e0-9349-39b23b8169bf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:56.657086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200cbdf-b708-44e0-9349-39b23b8169bf None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:56.659776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 691/1371] 10.222.0.48 () {54 vars in 964 bytes} [Tue Jul 26 17:04:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:56.669808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e49576-392f-4318-873f-ae670fb61db4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:56.670626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e49576-392f-4318-873f-ae670fb61db4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:56.670626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e49576-392f-4318-873f-ae670fb61db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:56.770965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-01e49576-392f-4318-873f-ae670fb61db4 None None] MFA Rules not processed for user `c2f2db6b70c241ec8bda9c96330f98c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:56.773072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-01e49576-392f-4318-873f-ae670fb61db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:56.865015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:56.925392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 681/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Tue Jul 26 17:04:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:04:56.940410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200cbdf-b708-44e0-9349-39b23b8169bf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:56.942729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4200cbdf-b708-44e0-9349-39b23b8169bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:57.089859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200cbdf-b708-44e0-9349-39b23b8169bf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:57.142540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:57.190226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:57.190778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:57.191066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:57.191348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:57.193167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:57.393080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:57.396109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c3a98-e1b7-4652-b5e9-2651fa5b1d0f service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:57.435524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 692/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:04:58.591462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973d9f-f658-4661-9709-fc1325cfc0cc None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:58.592024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973d9f-f658-4661-9709-fc1325cfc0cc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:58.592024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3973d9f-f658-4661-9709-fc1325cfc0cc None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:58.593566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 682/1374] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:04:58.599345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48dfdc-4f9e-46ef-b1e8-480bd5c4d20f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:58.600356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48dfdc-4f9e-46ef-b1e8-480bd5c4d20f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:58.601424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48dfdc-4f9e-46ef-b1e8-480bd5c4d20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:58.685893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2d48dfdc-4f9e-46ef-b1e8-480bd5c4d20f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:04:58.687153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2d48dfdc-4f9e-46ef-b1e8-480bd5c4d20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:58.714016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:58.809626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 693/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:04:58.857120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973d9f-f658-4661-9709-fc1325cfc0cc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:04:58.858657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a3973d9f-f658-4661-9709-fc1325cfc0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:59.056390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3973d9f-f658-4661-9709-fc1325cfc0cc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:04:59.120666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:04:59.199155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:04:59.199785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:04:59.200061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:04:59.200345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:04:59.202084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:04:59.320515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:04:59.322224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26211878-dccc-4544-b879-ecb47cc85da2 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:04:59.357467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 683/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:01.724273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661c57f-dc48-4cdb-8dbf-8c14d9a5ba76 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:01.725697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661c57f-dc48-4cdb-8dbf-8c14d9a5ba76 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:01.726456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661c57f-dc48-4cdb-8dbf-8c14d9a5ba76 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:01.729363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 694/1377] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:01.737723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c3ea6-991c-4aea-8dad-f4d1529e2119 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:01.738264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c3ea6-991c-4aea-8dad-f4d1529e2119 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:01.738264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c3ea6-991c-4aea-8dad-f4d1529e2119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:01.848284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6e2c3ea6-991c-4aea-8dad-f4d1529e2119 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:01.851735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6e2c3ea6-991c-4aea-8dad-f4d1529e2119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:01.879473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:02.031550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 684/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:02.079418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661c57f-dc48-4cdb-8dbf-8c14d9a5ba76 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:02.081096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0661c57f-dc48-4cdb-8dbf-8c14d9a5ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:02.273495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661c57f-dc48-4cdb-8dbf-8c14d9a5ba76 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:02.278188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:02.484522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:02.485466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:02.485763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:02.486009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:02.487800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:02.659533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:02.662678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb406f-00fb-4e76-a20e-66b47fb99d17 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:02.695619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 695/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:05.599530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeb32-ada5-45ca-9aea-ecbeefc0d13c None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:05.600361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeb32-ada5-45ca-9aea-ecbeefc0d13c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:05.600508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeb32-ada5-45ca-9aea-ecbeefc0d13c None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:05.602173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 685/1380] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:05.608840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e87d66-9b90-4e8b-afcd-caa79e4a8eb3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:05.609744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e87d66-9b90-4e8b-afcd-caa79e4a8eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:05.609744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e87d66-9b90-4e8b-afcd-caa79e4a8eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:05.624542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:05.704889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-72e87d66-9b90-4e8b-afcd-caa79e4a8eb3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:05.706605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-72e87d66-9b90-4e8b-afcd-caa79e4a8eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:05.861425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 696/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:05.910202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aeb32-ada5-45ca-9aea-ecbeefc0d13c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:05.912036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fb0aeb32-ada5-45ca-9aea-ecbeefc0d13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:06.119399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:06.136739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aeb32-ada5-45ca-9aea-ecbeefc0d13c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:06.300399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:06.301271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:06.301566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:06.301822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:06.304152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:06.463072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:06.465327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7e68a9-c7c1-4ebf-b9b4-f3c26bb9e75b service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:06.485700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:06.505160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 686/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:09.426384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f589-7b0b-4be2-b564-056909a0f4ad None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:09.427388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f589-7b0b-4be2-b564-056909a0f4ad None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:09.427388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b36f589-7b0b-4be2-b564-056909a0f4ad None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:09.428856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 697/1383] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:09.435162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9bdd5-c299-460f-b582-e84ad50123bb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:09.436230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9bdd5-c299-460f-b582-e84ad50123bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:09.436230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9bdd5-c299-460f-b582-e84ad50123bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:09.550465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-83d9bdd5-c299-460f-b582-e84ad50123bb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:09.552340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-83d9bdd5-c299-460f-b582-e84ad50123bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:09.747502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 687/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:09.800146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f589-7b0b-4be2-b564-056909a0f4ad None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:09.802043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6b36f589-7b0b-4be2-b564-056909a0f4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:09.943567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:10.053667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b36f589-7b0b-4be2-b564-056909a0f4ad None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:10.252431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:10.253373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:10.254225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:10.254744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:10.258854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:10.334124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:10.401987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:10.404778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22fe23-491f-4b27-bb80-8ad47a235df7 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:10.437872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 698/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:13.217079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac081c-067a-41e2-b467-cb1b261f4317 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:13.217643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac081c-067a-41e2-b467-cb1b261f4317 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:13.217643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac081c-067a-41e2-b467-cb1b261f4317 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:13.218984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 688/1386] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:13.224821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af83267-7bde-48dd-8c0d-3caf428365e3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:13.225988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af83267-7bde-48dd-8c0d-3caf428365e3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:13.225988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af83267-7bde-48dd-8c0d-3caf428365e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:13.307906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1af83267-7bde-48dd-8c0d-3caf428365e3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:13.309204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1af83267-7bde-48dd-8c0d-3caf428365e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:13.425934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 699/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:13.473513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac081c-067a-41e2-b467-cb1b261f4317 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:13.475056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a0ac081c-067a-41e2-b467-cb1b261f4317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:13.568501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:13.654437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac081c-067a-41e2-b467-cb1b261f4317 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:13.798503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:13.799041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:13.799325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:13.799601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:13.801393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:13.852446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:13.915976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:13.917681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac475fd-759f-42b3-bd9d-2e189f6d6a8d service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:13.939865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 689/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:15.830500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae40ce-61b1-4f92-88bd-f4d3c908c129 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:15.831314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae40ce-61b1-4f92-88bd-f4d3c908c129 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:15.831497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae40ce-61b1-4f92-88bd-f4d3c908c129 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:15.835349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 700/1389] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:15.841166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b55acc-d1d9-46e3-9340-8c911ed94269 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:15.841939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b55acc-d1d9-46e3-9340-8c911ed94269 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:15.841939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b55acc-d1d9-46e3-9340-8c911ed94269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:15.933730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-74b55acc-d1d9-46e3-9340-8c911ed94269 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:15.935152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-74b55acc-d1d9-46e3-9340-8c911ed94269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:16.118767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 690/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:05:16.232408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17922e-1e66-4f15-8d4d-6e2f0bdba9f8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:16.233075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17922e-1e66-4f15-8d4d-6e2f0bdba9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:16.233587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17922e-1e66-4f15-8d4d-6e2f0bdba9f8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:16.235545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 701/1391] 10.222.0.48 () {54 vars in 966 bytes} [Tue Jul 26 17:05:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:16.243533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:16.243989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:16.244889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:16.260080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d7568-b14c-411f-b41e-d52a40b50b7a None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:16.260705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d7568-b14c-411f-b41e-d52a40b50b7a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:16.261208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d7568-b14c-411f-b41e-d52a40b50b7a None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:16.267952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:16.268221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 702/1392] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:16.277030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:16.277428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:16.277900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:16.340051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:16.378519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] MFA Rules not processed for user `0f33d17889ee4fe3a9692abe24bbdea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:16.380065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:16.394100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:16.395917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:16.534859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 691/1393] 10.222.0.48 () {58 vars in 1063 bytes} [Tue Jul 26 17:05:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:16.546821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:16.548837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 703/1394] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:05:16.560267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aaf957-ed50-4f89-8482-54725b41ebc0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:16.600715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:16.602267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:16.702394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:16.703354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:16.703854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:16.704358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:16.707867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:16.760275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:16.786203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:16.805671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe7b78-5206-4218-8a74-36775b3d8942 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:16.896851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:16.899599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b32b29-6b85-454b-8be1-af81606b4666 service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:16.932444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 692/1395] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:16.996377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:16.997232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:16.997425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:17.000048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:17.003465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:17.171450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:17.173823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c97464-827a-433a-b6e8-895a343a2a3e service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:17.203156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:17.209715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 704/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:05:17.876678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fb08d-31cc-4811-877f-b093005ccb18 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:17.877208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fb08d-31cc-4811-877f-b093005ccb18 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:17.877549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fb08d-31cc-4811-877f-b093005ccb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:17.998361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e19fb08d-31cc-4811-877f-b093005ccb18 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:18.000467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e19fb08d-31cc-4811-877f-b093005ccb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:18.157096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:18.166307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 693/1397] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:05:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:18.228064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c719c86-a84e-42ff-ae74-148a8b157be8 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:18.228934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c719c86-a84e-42ff-ae74-148a8b157be8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:18.228934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c719c86-a84e-42ff-ae74-148a8b157be8 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:18.230419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 705/1398] 10.222.0.48 () {54 vars in 943 bytes} [Tue Jul 26 17:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:18.239498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207989a3-df59-4601-8d10-ee50e0fc8a31 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:18.240009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207989a3-df59-4601-8d10-ee50e0fc8a31 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:18.240504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207989a3-df59-4601-8d10-ee50e0fc8a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:18.337194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-207989a3-df59-4601-8d10-ee50e0fc8a31 None None] MFA Rules not processed for user `0bfb7deb3aaf48a6992edb1b01b17aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:18.338644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-207989a3-df59-4601-8d10-ee50e0fc8a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:18.496165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 694/1399] 10.222.0.48 () {58 vars in 1040 bytes} [Tue Jul 26 17:05:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:18.511970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c719c86-a84e-42ff-ae74-148a8b157be8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:18.513898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0c719c86-a84e-42ff-ae74-148a8b157be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:18.730967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c719c86-a84e-42ff-ae74-148a8b157be8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:18.880908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:18.913394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:18.914448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:18.914958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:18.915643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:18.918861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:19.064675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:19.066348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ece53e-8845-43f6-907a-213feb3a5109 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:19.076321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 706/1400] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:05:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:05:19.774043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520bd2-cefc-4358-af09-17a51f053130 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:19.774569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520bd2-cefc-4358-af09-17a51f053130 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:19.775038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520bd2-cefc-4358-af09-17a51f053130 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:19.776724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 695/1401] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:19.783011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57187b27-76c6-4c99-94fc-820d9a64ea94 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:19.783624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57187b27-76c6-4c99-94fc-820d9a64ea94 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:19.783624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57187b27-76c6-4c99-94fc-820d9a64ea94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:19.879077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-57187b27-76c6-4c99-94fc-820d9a64ea94 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:19.881742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-57187b27-76c6-4c99-94fc-820d9a64ea94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:19.974213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:20.028066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 707/1402] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:05:20.073681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc520bd2-cefc-4358-af09-17a51f053130 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:20.075602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc520bd2-cefc-4358-af09-17a51f053130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:20.180771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:20.335122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc520bd2-cefc-4358-af09-17a51f053130 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:20.572869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:20.573736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:20.574158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:20.574576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:20.576780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:20.616609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:20.774689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:20.778552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae47c2-aad4-4bed-9258-fe7cc2d828d3 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:20.832288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 696/1403] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 760 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:05:23.631206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9c82-89a8-4985-b90a-a1f1a6093234 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:23.632025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9c82-89a8-4985-b90a-a1f1a6093234 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:23.632025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9c82-89a8-4985-b90a-a1f1a6093234 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:23.633222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 708/1404] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:23.640077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b70db-845c-48d4-8818-8f4d1737c954 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:23.640672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b70db-845c-48d4-8818-8f4d1737c954 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:23.641459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b70db-845c-48d4-8818-8f4d1737c954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:23.770521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5a9b70db-845c-48d4-8818-8f4d1737c954 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:23.773975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5a9b70db-845c-48d4-8818-8f4d1737c954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:23.975632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:23.989043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 697/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:05:24.063352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9c82-89a8-4985-b90a-a1f1a6093234 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:24.066267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-99be9c82-89a8-4985-b90a-a1f1a6093234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:24.338911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9c82-89a8-4985-b90a-a1f1a6093234 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:24.431960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:24.534614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:24.535357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:24.535765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:24.536128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:24.538285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:24.698956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:24.700524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb5922-cc2a-4c0a-a60c-0d2521826b97 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:24.722642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 709/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:27.817161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67ccb2-b2fd-46d2-8301-2a05f9104c67 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:27.817701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67ccb2-b2fd-46d2-8301-2a05f9104c67 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:27.817987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67ccb2-b2fd-46d2-8301-2a05f9104c67 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:27.819804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 698/1407] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:27.826705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f436c-57e2-4ebf-b1d4-3e4b5cf1ef2e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:27.827597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f436c-57e2-4ebf-b1d4-3e4b5cf1ef2e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:27.827894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f436c-57e2-4ebf-b1d4-3e4b5cf1ef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:27.942652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c48f436c-57e2-4ebf-b1d4-3e4b5cf1ef2e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:27.944006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c48f436c-57e2-4ebf-b1d4-3e4b5cf1ef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:27.961458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:28.098169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 710/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:28.184561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67ccb2-b2fd-46d2-8301-2a05f9104c67 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:28.187668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0b67ccb2-b2fd-46d2-8301-2a05f9104c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:28.489031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67ccb2-b2fd-46d2-8301-2a05f9104c67 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:28.596314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:28.677095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:28.680231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:28.680933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:28.681391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:28.684205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:28.870239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:28.872860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563b4c1-8626-4c9f-8056-63d6509c5fa1 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:28.911159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 699/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:38.094330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=37 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:38.905951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=65 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:44.509309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac6e6b-657d-49f9-940f-726293dcb007 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:44.509894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac6e6b-657d-49f9-940f-726293dcb007 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:44.510101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac6e6b-657d-49f9-940f-726293dcb007 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:44.511610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 711/1410] 10.222.0.48 () {54 vars in 918 bytes} [Tue Jul 26 17:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:44.517797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dd374-8366-460a-a4e8-793408d03590 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:44.518339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dd374-8366-460a-a4e8-793408d03590 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:44.518339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dd374-8366-460a-a4e8-793408d03590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:44.628624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-348dd374-8366-460a-a4e8-793408d03590 None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:44.630937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-348dd374-8366-460a-a4e8-793408d03590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:44.774548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 700/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Tue Jul 26 17:05:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:05:44.797399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1d72-b9aa-4d9b-8760-f5208f8a2e6e None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:44.797906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1d72-b9aa-4d9b-8760-f5208f8a2e6e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:44.798314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1d72-b9aa-4d9b-8760-f5208f8a2e6e None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:44.800589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 712/1412] 10.222.0.48 () {54 vars in 918 bytes} [Tue Jul 26 17:05:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:44.808740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae668-ccb1-4241-bbd8-dd21bfd57324 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:44.809265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae668-ccb1-4241-bbd8-dd21bfd57324 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:44.809654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae668-ccb1-4241-bbd8-dd21bfd57324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:44.901137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:44.906542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6aeae668-ccb1-4241-bbd8-dd21bfd57324 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:44.908855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6aeae668-ccb1-4241-bbd8-dd21bfd57324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:45.054100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 701/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Tue Jul 26 17:05:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:47.880056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbeb5c3-a3af-42be-9e6f-d9b9ec634202 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:47.880739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbeb5c3-a3af-42be-9e6f-d9b9ec634202 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:47.880739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbeb5c3-a3af-42be-9e6f-d9b9ec634202 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:47.882131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:47.891857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb24585-be36-4198-8cfe-8081f282b743 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:47.892460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb24585-be36-4198-8cfe-8081f282b743 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:47.892855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb24585-be36-4198-8cfe-8081f282b743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:48.019612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cfb24585-be36-4198-8cfe-8081f282b743 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:48.022228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cfb24585-be36-4198-8cfe-8081f282b743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:48.106077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:48.166064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:48.213063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbeb5c3-a3af-42be-9e6f-d9b9ec634202 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:48.214541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8fbeb5c3-a3af-42be-9e6f-d9b9ec634202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:48.403911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbeb5c3-a3af-42be-9e6f-d9b9ec634202 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:48.528185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:48.592294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:48.592919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:48.593229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:48.593504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:48.595224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:48.709792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:48.711372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32ab65-4db3-4b28-a3d0-b504868e9a28 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:48.734421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 714/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:49.780250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8b4c-d2ea-40b8-8e7a-866f1c3ea4da None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:49.781006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8b4c-d2ea-40b8-8e7a-866f1c3ea4da None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:49.781006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe8b4c-d2ea-40b8-8e7a-866f1c3ea4da None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:49.782170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:49.789031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128109f0-170b-409f-b009-6953b00b1572 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:49.789719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128109f0-170b-409f-b009-6953b00b1572 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:49.789719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128109f0-170b-409f-b009-6953b00b1572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:49.888403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-128109f0-170b-409f-b009-6953b00b1572 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:49.889735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-128109f0-170b-409f-b009-6953b00b1572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:49.930819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:50.026841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:50.329347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8b4c-d2ea-40b8-8e7a-866f1c3ea4da None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:50.330939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-eefe8b4c-d2ea-40b8-8e7a-866f1c3ea4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:50.624263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe8b4c-d2ea-40b8-8e7a-866f1c3ea4da None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:50.706612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:50.801426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:50.802567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:50.802567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:50.802940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:50.804459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:50.919654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:50.921440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931f018-9c7a-4fd8-bd3b-45e77599bb17 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:50.945235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 704/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:05:51.924783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861909f-c625-44cf-aefb-22d79e536d9b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:51.925435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861909f-c625-44cf-aefb-22d79e536d9b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:51.925435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861909f-c625-44cf-aefb-22d79e536d9b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:51.926757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:51.933570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60c6a-9611-42d3-9db2-3bfe1e1c1219 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:51.934394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60c6a-9611-42d3-9db2-3bfe1e1c1219 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:51.934394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60c6a-9611-42d3-9db2-3bfe1e1c1219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:52.007501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:52.039111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-38c60c6a-9611-42d3-9db2-3bfe1e1c1219 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:52.040383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-38c60c6a-9611-42d3-9db2-3bfe1e1c1219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:52.166655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:05:52.342681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861909f-c625-44cf-aefb-22d79e536d9b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:52.345568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c861909f-c625-44cf-aefb-22d79e536d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:52.553348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861909f-c625-44cf-aefb-22d79e536d9b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:52.571351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:52.736435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:52.737627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:52.738336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:52.738853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:52.743360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:52.888185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:52.891580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed0748-e5a8-4f8f-b736-fd96088942ef service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:52.917144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 717/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:54.006923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f57cb-4f5f-409d-83bf-6de5c43417df None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:54.007643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f57cb-4f5f-409d-83bf-6de5c43417df None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:54.007643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f57cb-4f5f-409d-83bf-6de5c43417df None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:54.008921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:05:54.017327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6588-6f75-4d29-8e98-d3f151e61162 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:54.018197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6588-6f75-4d29-8e98-d3f151e61162 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:54.018197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe6588-6f75-4d29-8e98-d3f151e61162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:54.048514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:54.126356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-76fe6588-6f75-4d29-8e98-d3f151e61162 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:54.127536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-76fe6588-6f75-4d29-8e98-d3f151e61162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:54.260048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:54.428194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f57cb-4f5f-409d-83bf-6de5c43417df None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:54.429983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1e7f57cb-4f5f-409d-83bf-6de5c43417df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:54.572800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:54.625912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f57cb-4f5f-409d-83bf-6de5c43417df None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:54.901594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:54.902235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:54.902636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:54.902811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:54.904584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:54.997388 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:55.014327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:55.015950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3f370-f656-496a-9bb5-c59a82b8c2b6 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:55.038344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 707/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:05:56.035123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e933aff-0b5b-4820-aa7a-7211f0c962b7 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:56.035839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e933aff-0b5b-4820-aa7a-7211f0c962b7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:56.035839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e933aff-0b5b-4820-aa7a-7211f0c962b7 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:56.038238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:56.043809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7993-f8f0-444a-8d36-303596424ce2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:56.044381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7993-f8f0-444a-8d36-303596424ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:56.044528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7993-f8f0-444a-8d36-303596424ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:56.144563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b0ae7993-f8f0-444a-8d36-303596424ce2 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:56.147190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b0ae7993-f8f0-444a-8d36-303596424ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:56.322174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:56.486939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e933aff-0b5b-4820-aa7a-7211f0c962b7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:56.488734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4e933aff-0b5b-4820-aa7a-7211f0c962b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:56.603698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:56.695831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e933aff-0b5b-4820-aa7a-7211f0c962b7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:56.834773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:56.835662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:56.836067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:56.836440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:56.838352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:56.903531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:56.954703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:56.956457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcfb89-d1bf-4feb-94fb-aa696b1c14d7 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:56.982349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 720/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:57.832157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a918b5b-7e29-48d1-80a8-290d1e94db23 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:57.833080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a918b5b-7e29-48d1-80a8-290d1e94db23 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:57.833274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a918b5b-7e29-48d1-80a8-290d1e94db23 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:57.835784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:57.841933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645abdbb-80e6-4000-878d-59a5fa442142 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:57.842626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645abdbb-80e6-4000-878d-59a5fa442142 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:57.842626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645abdbb-80e6-4000-878d-59a5fa442142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:57.965716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-645abdbb-80e6-4000-878d-59a5fa442142 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:57.967849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-645abdbb-80e6-4000-878d-59a5fa442142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:58.140942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:05:58.305682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a918b5b-7e29-48d1-80a8-290d1e94db23 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:05:58.307363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3a918b5b-7e29-48d1-80a8-290d1e94db23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:58.367884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:58.496723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a918b5b-7e29-48d1-80a8-290d1e94db23 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:05:58.635523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:05:58.636156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:58.636467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:58.636714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:58.638677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:58.647313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:58.803163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:05:58.806854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c2cf3-054e-4d8d-b7a3-87ad9a3ee17f service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:05:58.842932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 710/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:05:59.739898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50045438-1c80-4a07-90f1-4b6cdee0c4a1 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:59.740437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50045438-1c80-4a07-90f1-4b6cdee0c4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:59.740437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50045438-1c80-4a07-90f1-4b6cdee0c4a1 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:59.741892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:05:59.749685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63711d-dde5-492a-8180-7d0e441258cf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:05:59.750146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63711d-dde5-492a-8180-7d0e441258cf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:05:59.750520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63711d-dde5-492a-8180-7d0e441258cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:05:59.840045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3d63711d-dde5-492a-8180-7d0e441258cf None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:05:59.841333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3d63711d-dde5-492a-8180-7d0e441258cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:05:59.929641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:05:59.968237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:00.132595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50045438-1c80-4a07-90f1-4b6cdee0c4a1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:00.134246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-50045438-1c80-4a07-90f1-4b6cdee0c4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:00.158430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:00.336833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50045438-1c80-4a07-90f1-4b6cdee0c4a1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:00.458573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:00.513044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:00.514084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:00.514569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:00.515098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:00.518469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:00.690287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:00.691884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593034c-5881-47e0-9536-5d6676c9fc49 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:00.713911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 723/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:06:01.622572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8215fd-f769-4b41-91c1-3b52d784d139 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:01.623585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8215fd-f769-4b41-91c1-3b52d784d139 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:01.623585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8215fd-f769-4b41-91c1-3b52d784d139 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:01.624781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:01.631122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2e49b-8c13-41b8-a2a3-27a64b719e6e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:01.632091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2e49b-8c13-41b8-a2a3-27a64b719e6e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:01.632091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2e49b-8c13-41b8-a2a3-27a64b719e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:01.722937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-19f2e49b-8c13-41b8-a2a3-27a64b719e6e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:01.724201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-19f2e49b-8c13-41b8-a2a3-27a64b719e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:01.783985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:01.888420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:02.049780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8215fd-f769-4b41-91c1-3b52d784d139 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:02.051241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8b8215fd-f769-4b41-91c1-3b52d784d139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:02.228623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8215fd-f769-4b41-91c1-3b52d784d139 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:02.295998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:02.410337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:02.411026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:02.411352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:02.411716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:02.413684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:02.521382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:02.523120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164fc7d-f302-49f1-b39c-f2558be9dbf9 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:02.546634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 713/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:03.409803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a2e25-dc60-4c05-a1ef-b3a8ace6d536 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:03.410355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a2e25-dc60-4c05-a1ef-b3a8ace6d536 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:03.410484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a2e25-dc60-4c05-a1ef-b3a8ace6d536 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:03.412053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:03.418353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be669538-6d7e-41ba-842c-fbc92e1f76b0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:03.418969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be669538-6d7e-41ba-842c-fbc92e1f76b0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:03.418969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be669538-6d7e-41ba-842c-fbc92e1f76b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:03.511125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-be669538-6d7e-41ba-842c-fbc92e1f76b0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:03.512392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-be669538-6d7e-41ba-842c-fbc92e1f76b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:03.514897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:03.634492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:06:03.790673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a2e25-dc60-4c05-a1ef-b3a8ace6d536 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:03.792143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8e6a2e25-dc60-4c05-a1ef-b3a8ace6d536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:03.975260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a2e25-dc60-4c05-a1ef-b3a8ace6d536 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:04.008981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:04.120446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:04.121285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:04.121563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:04.121837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:04.123589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:04.233551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:04.235252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fe17c-5ac4-4e1e-9138-78b7b69ba87a service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:04.258642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 726/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:06.285888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d93ec-1bef-4abe-b7a2-7ed3739baa70 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:06.286450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d93ec-1bef-4abe-b7a2-7ed3739baa70 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:06.286450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d93ec-1bef-4abe-b7a2-7ed3739baa70 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:06.288205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:06.294776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bd42f-d52b-432a-9962-16a05eabf17a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:06.295570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bd42f-d52b-432a-9962-16a05eabf17a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:06.295570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bd42f-d52b-432a-9962-16a05eabf17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:06.345443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:06.378264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-886bd42f-d52b-432a-9962-16a05eabf17a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:06.379554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-886bd42f-d52b-432a-9962-16a05eabf17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:06.505778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:06:06.548109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d93ec-1bef-4abe-b7a2-7ed3739baa70 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:06.549711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aa4d93ec-1bef-4abe-b7a2-7ed3739baa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:06.679859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d93ec-1bef-4abe-b7a2-7ed3739baa70 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:06.720126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:06.788650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:06.789767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:06.790266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:06.790768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:06.794260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:06.975751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:06.977231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d6adc-fa92-447f-aef1-6fba481ea997 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:07.003411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 716/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:06:09.495812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736805c9-f83e-4630-9065-bcfedf4aa255 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:09.496636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736805c9-f83e-4630-9065-bcfedf4aa255 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:09.496887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736805c9-f83e-4630-9065-bcfedf4aa255 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:09.498706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:09.506906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034ff08-4030-41f4-94f3-9d456aff5d45 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:09.507524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034ff08-4030-41f4-94f3-9d456aff5d45 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:09.507884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e034ff08-4030-41f4-94f3-9d456aff5d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:09.621324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e034ff08-4030-41f4-94f3-9d456aff5d45 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:09.622594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e034ff08-4030-41f4-94f3-9d456aff5d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:09.639284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:09.764373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:09.806466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736805c9-f83e-4630-9065-bcfedf4aa255 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:09.809623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-736805c9-f83e-4630-9065-bcfedf4aa255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:09.961108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736805c9-f83e-4630-9065-bcfedf4aa255 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:09.976157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:10.054749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:10.055294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:10.055575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:10.055848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:10.057647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:10.201586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:10.204378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da1067-3722-4ddd-b222-f0aea56b181d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:10.236046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 729/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:13.138482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a07302-8ae9-4d17-94bf-82c6f0eeba16 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:13.139082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a07302-8ae9-4d17-94bf-82c6f0eeba16 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:13.139082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a07302-8ae9-4d17-94bf-82c6f0eeba16 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:13.140688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:13.147683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acb3cd-43d9-4e84-84fe-f20ddbfe63b2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:13.148302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acb3cd-43d9-4e84-84fe-f20ddbfe63b2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:13.148302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acb3cd-43d9-4e84-84fe-f20ddbfe63b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:13.210449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:13.234371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-31acb3cd-43d9-4e84-84fe-f20ddbfe63b2 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:13.236111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-31acb3cd-43d9-4e84-84fe-f20ddbfe63b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:13.356427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:13.398474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a07302-8ae9-4d17-94bf-82c6f0eeba16 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:13.400156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-60a07302-8ae9-4d17-94bf-82c6f0eeba16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:13.578559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a07302-8ae9-4d17-94bf-82c6f0eeba16 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:13.629591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:13.671766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:13.672544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:13.672924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:13.673326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:13.675335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:13.835586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:13.838322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71193a47-e552-439e-9b69-336f6f622b64 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:13.875627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 719/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:06:16.191217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4b06b-45b3-46d3-974c-94b5ad896aea None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:16.191802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4b06b-45b3-46d3-974c-94b5ad896aea None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:16.191802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4b06b-45b3-46d3-974c-94b5ad896aea None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:16.194395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:06:16.201384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50996e-cb7b-4724-bfa2-b4725c01bbbb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:16.202251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50996e-cb7b-4724-bfa2-b4725c01bbbb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:16.202251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50996e-cb7b-4724-bfa2-b4725c01bbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:16.283671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cb50996e-cb7b-4724-bfa2-b4725c01bbbb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:16.284933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cb50996e-cb7b-4724-bfa2-b4725c01bbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:16.311939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:16.434366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:06:16.478246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4b06b-45b3-46d3-974c-94b5ad896aea None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:16.479953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3be4b06b-45b3-46d3-974c-94b5ad896aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:16.618194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4b06b-45b3-46d3-974c-94b5ad896aea None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:16.642761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:16.714205 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:16.714906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:16.715135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:16.715609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:16.717706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:16.854700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:16.856203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0cc10-d185-453b-8003-42793f56e68a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:16.882905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 732/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:06:19.225457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9eb69-a914-4f65-9619-9922a6a8cd9d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:19.226217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9eb69-a914-4f65-9619-9922a6a8cd9d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:19.226217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9eb69-a914-4f65-9619-9922a6a8cd9d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:19.227932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:06:19.234473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1453c41-e366-428e-a920-e499abc15bf8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:19.235292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1453c41-e366-428e-a920-e499abc15bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:19.235292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1453c41-e366-428e-a920-e499abc15bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:19.317928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:19.321862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f1453c41-e366-428e-a920-e499abc15bf8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:19.323844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1453c41-e366-428e-a920-e499abc15bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:19.455596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:06:19.498298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9eb69-a914-4f65-9619-9922a6a8cd9d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:19.499901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f2a9eb69-a914-4f65-9619-9922a6a8cd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:19.678330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9eb69-a914-4f65-9619-9922a6a8cd9d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:19.769138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:19.809093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:19.809767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:19.810053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:19.810335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:19.812087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:19.956205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:19.958500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65550-4231-480c-b596-9810f8a616f9 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:19.989418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 722/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:06:22.352775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759ba88-70db-4b2d-8c32-19bd9dfb873b None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:22.353367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759ba88-70db-4b2d-8c32-19bd9dfb873b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:22.353489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5759ba88-70db-4b2d-8c32-19bd9dfb873b None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:22.355108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:22.364019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3522ed-1729-428f-8fc6-9df02ab1e3d6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:22.364019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3522ed-1729-428f-8fc6-9df02ab1e3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:22.365526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3522ed-1729-428f-8fc6-9df02ab1e3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:22.465844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fb3522ed-1729-428f-8fc6-9df02ab1e3d6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:22.467138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fb3522ed-1729-428f-8fc6-9df02ab1e3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:22.503335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:22.605559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:22.650889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759ba88-70db-4b2d-8c32-19bd9dfb873b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:22.652511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5759ba88-70db-4b2d-8c32-19bd9dfb873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:22.788840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5759ba88-70db-4b2d-8c32-19bd9dfb873b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:22.850723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:22.950332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:22.951324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:22.951786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:22.952225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:22.955250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:23.150281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:23.153065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61a029-c63e-4528-9b30-39f617b652f5 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:23.186996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 735/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:25.714696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f847-2e7f-43eb-93ed-e0bcb57caadf None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:25.715319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f847-2e7f-43eb-93ed-e0bcb57caadf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:25.715319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74f847-2e7f-43eb-93ed-e0bcb57caadf None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:25.716882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:06:25.723182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa124e51-3199-4089-af56-dfe9a1a46d80 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:25.724672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa124e51-3199-4089-af56-dfe9a1a46d80 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:25.724672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa124e51-3199-4089-af56-dfe9a1a46d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:25.814114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-aa124e51-3199-4089-af56-dfe9a1a46d80 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:25.815867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa124e51-3199-4089-af56-dfe9a1a46d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:25.828657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:25.996939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:26.063825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74f847-2e7f-43eb-93ed-e0bcb57caadf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:26.065492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ff74f847-2e7f-43eb-93ed-e0bcb57caadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:26.198753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74f847-2e7f-43eb-93ed-e0bcb57caadf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:26.274596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:26.312121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:26.313230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:26.313704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:26.314183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:26.318003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:26.503907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:26.506700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cd1d6-3946-4ca4-9951-f76825650b39 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:26.531981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 725/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:28.836665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f226a0-5b3c-48a4-a156-316209d8b693 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:28.838507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f226a0-5b3c-48a4-a156-316209d8b693 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:28.838507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f226a0-5b3c-48a4-a156-316209d8b693 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:28.839005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:06:28.844891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea58f5-58fb-4ae8-8174-ba9b64401024 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:28.845600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea58f5-58fb-4ae8-8174-ba9b64401024 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:28.845600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea58f5-58fb-4ae8-8174-ba9b64401024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:28.953224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d8ea58f5-58fb-4ae8-8174-ba9b64401024 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:28.955289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d8ea58f5-58fb-4ae8-8174-ba9b64401024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:29.032291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:29.119471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:29.160537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f226a0-5b3c-48a4-a156-316209d8b693 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:29.162278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-00f226a0-5b3c-48a4-a156-316209d8b693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:29.287979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f226a0-5b3c-48a4-a156-316209d8b693 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:29.332891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:29.376086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:29.376725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:29.377028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:29.377302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:29.379013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:29.519289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:29.520897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec46e8-fd6a-4e65-ad01-bddc21b060c7 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:29.544528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 738/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:32.499320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba7c8b-beb4-493f-8555-00496506eab8 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:32.499905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba7c8b-beb4-493f-8555-00496506eab8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:32.499905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba7c8b-beb4-493f-8555-00496506eab8 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:32.501284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:32.507591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20888bab-8334-4895-ae92-e89c90fc62a0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:32.508280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20888bab-8334-4895-ae92-e89c90fc62a0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:32.508280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20888bab-8334-4895-ae92-e89c90fc62a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:32.588098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-20888bab-8334-4895-ae92-e89c90fc62a0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:32.589419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-20888bab-8334-4895-ae92-e89c90fc62a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:32.610767 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:32.703557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:32.743047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba7c8b-beb4-493f-8555-00496506eab8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:32.744655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-60ba7c8b-beb4-493f-8555-00496506eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:32.866507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba7c8b-beb4-493f-8555-00496506eab8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:32.949411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:32.957714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:32.959067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:32.959356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:32.959629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:32.961418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:33.085929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:33.087408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe93f1-b6c1-4203-b55d-d54b19ce9228 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:33.108553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 728/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:35.363082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05b8ab-e30d-47af-b348-cd06d12b7b22 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:35.363706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05b8ab-e30d-47af-b348-cd06d12b7b22 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:35.363706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05b8ab-e30d-47af-b348-cd06d12b7b22 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:35.366705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:35.374202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09af05c-718c-40bb-92a9-c98481ad4022 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:35.375055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09af05c-718c-40bb-92a9-c98481ad4022 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:35.375348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09af05c-718c-40bb-92a9-c98481ad4022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:35.470543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a09af05c-718c-40bb-92a9-c98481ad4022 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:35.471963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a09af05c-718c-40bb-92a9-c98481ad4022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:35.539436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:35.622105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:06:35.666610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05b8ab-e30d-47af-b348-cd06d12b7b22 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:35.668627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5a05b8ab-e30d-47af-b348-cd06d12b7b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:35.808849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05b8ab-e30d-47af-b348-cd06d12b7b22 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:35.883313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:35.918064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:35.918728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:35.919018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:35.919283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:35.921063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:36.082013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:36.083925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d9701-ab9c-482a-8b65-0511366278ab service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:36.105493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 741/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:06:45.618693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=22 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:46.102983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=66 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:58.119799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cad6d-9d66-4869-ae83-e451d1b2a0ea None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:58.121328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cad6d-9d66-4869-ae83-e451d1b2a0ea None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:58.121828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cad6d-9d66-4869-ae83-e451d1b2a0ea None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:58.124054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 730/1471] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:06:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:58.132913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305e51b-0bd6-4de3-a639-7cb791ba1966 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:58.133579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305e51b-0bd6-4de3-a639-7cb791ba1966 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:58.133956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305e51b-0bd6-4de3-a639-7cb791ba1966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:58.234247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d305e51b-0bd6-4de3-a639-7cb791ba1966 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:58.235376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d305e51b-0bd6-4de3-a639-7cb791ba1966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:58.357293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 742/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:58.453923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8937958-5473-4616-bc56-3b2dc2280c45 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:58.454371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8937958-5473-4616-bc56-3b2dc2280c45 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:58.454767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8937958-5473-4616-bc56-3b2dc2280c45 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:58.456418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 731/1473] 10.222.0.48 () {54 vars in 966 bytes} [Tue Jul 26 17:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:06:58.462893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc541ab-c4d6-4c67-b170-01dedb14186e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:58.463228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc541ab-c4d6-4c67-b170-01dedb14186e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:58.463621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc541ab-c4d6-4c67-b170-01dedb14186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:58.558684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3bc541ab-c4d6-4c67-b170-01dedb14186e None None] MFA Rules not processed for user `0f33d17889ee4fe3a9692abe24bbdea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:06:58.559844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3bc541ab-c4d6-4c67-b170-01dedb14186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:58.586040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:58.656133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 743/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Tue Jul 26 17:06:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:06:58.664416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8937958-5473-4616-bc56-3b2dc2280c45 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:06:58.665983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a8937958-5473-4616-bc56-3b2dc2280c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:58.829681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8937958-5473-4616-bc56-3b2dc2280c45 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:06:58.933081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:06:58.933713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:06:58.933990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:06:58.934262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:06:58.936039 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:06:59.022756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:06:59.100733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:06:59.102241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26146092-9887-4b18-86d4-5d6276b16e3b service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:06:59.127677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 732/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:07:01.395818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523fdcd-85cc-454c-90a6-238957eae6b4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:01.396430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523fdcd-85cc-454c-90a6-238957eae6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:01.396538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523fdcd-85cc-454c-90a6-238957eae6b4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:01.398639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 744/1476] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:07:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:07:01.407505 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f25e-e2d9-46b0-a508-7dfa5aa6429e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:01.408094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f25e-e2d9-46b0-a508-7dfa5aa6429e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:01.408986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f25e-e2d9-46b0-a508-7dfa5aa6429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:01.496517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0d11f25e-e2d9-46b0-a508-7dfa5aa6429e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:07:01.497795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0d11f25e-e2d9-46b0-a508-7dfa5aa6429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:01.634032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 733/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:07:01.658924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523fdcd-85cc-454c-90a6-238957eae6b4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:07:01.661201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a523fdcd-85cc-454c-90a6-238957eae6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:01.866664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523fdcd-85cc-454c-90a6-238957eae6b4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:07:01.934263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:02.006179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:07:02.006821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:02.007104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:02.007374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:02.008090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:07:02.009982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:07:02.009982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:07:02.009982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:07:02.011165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7ec95-5e90-4798-a863-970e35477ece admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:07:02.035086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 745/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Tue Jul 26 17:07:01 2022] GET /identity/v3/projects => generated 2553 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:07:04.282000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d5e3b-d89e-47c4-bfe2-abad132946b9 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:04.282572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d5e3b-d89e-47c4-bfe2-abad132946b9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:04.282572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d5e3b-d89e-47c4-bfe2-abad132946b9 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:04.284033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 734/1479] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:07:04.290523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558efaea-6799-497f-a066-3dbb95a9bc85 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:04.291134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558efaea-6799-497f-a066-3dbb95a9bc85 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:04.291134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558efaea-6799-497f-a066-3dbb95a9bc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:04.376101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-558efaea-6799-497f-a066-3dbb95a9bc85 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:07:04.377396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-558efaea-6799-497f-a066-3dbb95a9bc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:04.497954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 746/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:07:04.546750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d5e3b-d89e-47c4-bfe2-abad132946b9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:07:04.548858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c22d5e3b-d89e-47c4-bfe2-abad132946b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:04.561010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:04.754041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d5e3b-d89e-47c4-bfe2-abad132946b9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:07:04.853412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:04.893004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:07:04.893681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:04.893964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:04.894288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:04.896372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:05.007377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:07:05.009046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e910d55-3153-4267-bbb5-f9824028c461 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:07:05.029873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 735/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:07:07.377558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b54803-9746-440f-83bc-d5b86f015cd0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:07.378188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b54803-9746-440f-83bc-d5b86f015cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:07.378188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b54803-9746-440f-83bc-d5b86f015cd0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:07.380094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 747/1482] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:07:07.386524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5289-947c-4336-a52d-fb67fb2ecc4f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:07.387399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5289-947c-4336-a52d-fb67fb2ecc4f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:07.387399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde5289-947c-4336-a52d-fb67fb2ecc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:07.476582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ccde5289-947c-4336-a52d-fb67fb2ecc4f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:07:07.477889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ccde5289-947c-4336-a52d-fb67fb2ecc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:07.533789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:07.609189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 736/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:07:07.649578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b54803-9746-440f-83bc-d5b86f015cd0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:07:07.651184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-84b54803-9746-440f-83bc-d5b86f015cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:07.679115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:07.819578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b54803-9746-440f-83bc-d5b86f015cd0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:07:07.946637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:07:07.947282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:07.947562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:07.947838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:07.949622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:08.070355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:08.138471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:07:08.141408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f07b-25c4-4fe9-b347-e85c730f3a9c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:07:08.167231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 748/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:07:10.392364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440913-749b-4ab5-8644-9d1c3d347b97 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:10.393136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440913-749b-4ab5-8644-9d1c3d347b97 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:10.393136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440913-749b-4ab5-8644-9d1c3d347b97 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:10.394375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 737/1485] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:07:10.401142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5ca44-cefc-4046-8351-ae9f593e4471 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:10.401710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5ca44-cefc-4046-8351-ae9f593e4471 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:10.401710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5ca44-cefc-4046-8351-ae9f593e4471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:10.484647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c8b5ca44-cefc-4046-8351-ae9f593e4471 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:07:10.485953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c8b5ca44-cefc-4046-8351-ae9f593e4471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:10.606687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 749/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:07:10.647248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440913-749b-4ab5-8644-9d1c3d347b97 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:07:10.648867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c9440913-749b-4ab5-8644-9d1c3d347b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:10.775732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440913-749b-4ab5-8644-9d1c3d347b97 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:07:10.875559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:10.878848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:07:10.881119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:10.881692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:10.882040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:10.885736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:11.050254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:07:11.052196 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fdc2a-c7a0-4fc4-8c94-73d160b00e68 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:07:11.074375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 738/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:07:13.521872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f062e-6055-4ea6-baf3-038fb5bd2620 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:13.522543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f062e-6055-4ea6-baf3-038fb5bd2620 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:13.522543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f062e-6055-4ea6-baf3-038fb5bd2620 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:13.523835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 750/1488] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:07:13.530078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774f73-40fe-4746-ba8e-ece0767e5113 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:13.530603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774f73-40fe-4746-ba8e-ece0767e5113 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:13.530703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34774f73-40fe-4746-ba8e-ece0767e5113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:13.608560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-34774f73-40fe-4746-ba8e-ece0767e5113 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:07:13.610051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-34774f73-40fe-4746-ba8e-ece0767e5113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:13.674575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:13.727667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 739/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:07:13.770072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f062e-6055-4ea6-baf3-038fb5bd2620 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:07:13.771871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-690f062e-6055-4ea6-baf3-038fb5bd2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:13.818263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:13.902106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f062e-6055-4ea6-baf3-038fb5bd2620 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:07:13.985549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:07:13.986095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:13.986371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:13.986912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:13.988816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:14.080319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:14.116441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:07:14.117978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1a40e-2688-4a9e-ac7e-c8595a0e88f1 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:07:14.139996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 751/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:07:16.554036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0cbfa-28fc-42d8-abc2-bf5bf6ca9107 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:16.554806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0cbfa-28fc-42d8-abc2-bf5bf6ca9107 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:16.554806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0cbfa-28fc-42d8-abc2-bf5bf6ca9107 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:16.556037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 740/1491] 10.222.0.48 () {54 vars in 926 bytes} [Tue Jul 26 17:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:07:16.562567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b854c0-6eca-4127-b807-86cf66b01c66 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:16.563117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b854c0-6eca-4127-b807-86cf66b01c66 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:16.563238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b854c0-6eca-4127-b807-86cf66b01c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:16.641475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-21b854c0-6eca-4127-b807-86cf66b01c66 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:07:16.642730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-21b854c0-6eca-4127-b807-86cf66b01c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:16.766042 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 752/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Tue Jul 26 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:07:16.808281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0cbfa-28fc-42d8-abc2-bf5bf6ca9107 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:07:16.810072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d9d0cbfa-28fc-42d8-abc2-bf5bf6ca9107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:16.944040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0cbfa-28fc-42d8-abc2-bf5bf6ca9107 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:07:17.038190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:07:17.038742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:07:17.039025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:07:17.039299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:07:17.041134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:07:17.055497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:17.186923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:07:17.188563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a36071-1fce-4c09-bb16-a5ee78d1aca0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:07:17.210163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 741/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:07:26.763750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=82 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:07:27.207603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=50 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:09:02.917786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:02.918548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:02.918771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.065888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:09:03.067181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:03.197714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 753/1494] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:09:03.206696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.207449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.207449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.243289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:09:03.245522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:03.262367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 742/1495] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:09:03.269392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:03.280352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32320-ccb4-4fa9-bacc-320c18f0bfed None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:03.286110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:03.286781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.287063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.287361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.288080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:09:03.288383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:03.289512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:03.289512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:09:03.289512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:09:03.290455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:03.305863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 754/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:09:03.315785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:03.327419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8577cf-a5f6-4cc9-82c3-4b5591885534 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:03.335350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:03.336277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.336699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.337174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.338393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:03.341488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:03.341488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:09:03.341488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:09:03.343526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:03.399563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 743/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:09:03 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:09:03.404467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:03.415858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69edce-fdc3-4855-a1ac-2c1a40de021a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:03.476491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:09:03.484702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:03.485387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.485670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.485945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.486699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:03.488156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:03.488156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:09:03.488156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:09:03.489710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:03.514241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 755/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:09:03 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:09:03.518905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:03.531231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ce01c-6ec4-43c8-8b63-9ca56024337f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:03.536927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:03.537614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.537907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.538181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.538901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:03.539935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:03.539935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:09:03.539935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:09:03.540839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:03.555765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 744/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:09:03.560455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:03.573151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200333cd-7987-49cf-af04-275d242d8a13 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:03.579092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:03.579751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.580037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.580808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.581584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:03.582662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:03.582662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:09:03.582662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:09:03.583595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83889556-8ef3-492d-b8d5-b89565028d77 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:03.600185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 756/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:09:03.604784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:03.616156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab156e9-8cea-403e-ace9-eaa7a3bc1108 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:03.621189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:03.621917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.622198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.622473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] PATH_INFO: `/v3/projects/913c2a4847ce48a08a2108ff7fdfd250/users/fb82274c2cde4039b86d2e163f1867cf/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.639899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913c2a4847ce48a08a2108ff7fdfd250, user_id=fb82274c2cde4039b86d2e163f1867cf, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:03.641087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:03.641087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:09:03.641087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:09:03.643641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32746d-ea5f-4355-9e3d-f3d37cdb7871 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:03.659862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 745/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:09:03 2022] PUT /identity/v3/projects/913c2a4847ce48a08a2108ff7fdfd250/users/fb82274c2cde4039b86d2e163f1867cf/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:09:03.674528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f263-d4dc-42dd-a4da-34b6a1747554 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.675039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f263-d4dc-42dd-a4da-34b6a1747554 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.675561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f263-d4dc-42dd-a4da-34b6a1747554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.721775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ef38f263-d4dc-42dd-a4da-34b6a1747554 None None] MFA Rules not processed for user `fb82274c2cde4039b86d2e163f1867cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:09:03.723159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ef38f263-d4dc-42dd-a4da-34b6a1747554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:03.800819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 757/1502] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:09:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:09:03.816688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263581b-3b97-4ca6-a370-860837d1af3d None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.817122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263581b-3b97-4ca6-a370-860837d1af3d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.817459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263581b-3b97-4ca6-a370-860837d1af3d None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.818913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 746/1503] 10.222.0.48 () {54 vars in 964 bytes} [Tue Jul 26 17:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:09:03.824706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4738-26c4-43fe-b4a2-15c7ac47beb3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:03.825035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4738-26c4-43fe-b4a2-15c7ac47beb3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:03.825314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4738-26c4-43fe-b4a2-15c7ac47beb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:03.898375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-adba4738-26c4-43fe-b4a2-15c7ac47beb3 None None] MFA Rules not processed for user `c2f2db6b70c241ec8bda9c96330f98c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:09:03.899606 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-adba4738-26c4-43fe-b4a2-15c7ac47beb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:03.996345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 758/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Tue Jul 26 17:09:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:09:04.005384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263581b-3b97-4ca6-a370-860837d1af3d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:04.017192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263581b-3b97-4ca6-a370-860837d1af3d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:04.022094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:04.022737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:04.023019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:04.023297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:04.033224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:04.034735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:04.045169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 747/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:09:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:09:04.099116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413ea7-83be-4518-8b37-2ae6d3d158e2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:04.099396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413ea7-83be-4518-8b37-2ae6d3d158e2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:04.099670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413ea7-83be-4518-8b37-2ae6d3d158e2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:04.101049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 759/1506] 10.222.0.48 () {52 vars in 830 bytes} [Tue Jul 26 17:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:09:04.105436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:04.183504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680234d9-aab1-429d-ae4c-9f0950209621 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:04.188487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:04.189203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:04.189487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:04.190325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] PATH_INFO: `/v3/users/fb82274c2cde4039b86d2e163f1867cf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:04.191817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb82274c2cde4039b86d2e163f1867cf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:04.192856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:04.192856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:09:04.192856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:09:04.194130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:04.274985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb82274c2cde4039b86d2e163f1867cf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:04.275987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb82274c2cde4039b86d2e163f1867cf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:04.298716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb82274c2cde4039b86d2e163f1867cf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:04.310344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb82274c2cde4039b86d2e163f1867cf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:04.321115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:09:04.327637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb82274c2cde4039b86d2e163f1867cf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:04.337153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 748/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:09:04 2022] DELETE /identity/v3/users/fb82274c2cde4039b86d2e163f1867cf => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:09:04.342100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:04.342472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:04.342788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:04.494103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:09:04.495355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:04.506621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:09:04.587153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 760/1508] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:09:04.598463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:04.600043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:04.701552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d1a55-25c1-4fb1-8787-f11f7b0c73b7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:04.705769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c60faa-adea-447c-a48b-12433b93e630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:04.706496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c60faa-adea-447c-a48b-12433b93e630 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:04.706809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c60faa-adea-447c-a48b-12433b93e630 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:04.707117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c60faa-adea-447c-a48b-12433b93e630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:04.718721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c60faa-adea-447c-a48b-12433b93e630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:04.720405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c60faa-adea-447c-a48b-12433b93e630 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:04.732226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 749/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:09:05.160278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:09:05.162769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:09:05.193067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176befbc-58f4-4037-b816-1d08381e1591 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:09:05.202669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:09:05.203784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:09:05.204283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:09:05.204782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] PATH_INFO: `/v3/projects/913c2a4847ce48a08a2108ff7fdfd250` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:09:05.208030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913c2a4847ce48a08a2108ff7fdfd250)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:09:05.209968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:09:05.209968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:09:05.209968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:09:05.212326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:09:05.305796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913c2a4847ce48a08a2108ff7fdfd250', 'request_id': 'req-89d45846-5f31-491f-9389-e7c57110a7d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:05.321210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913c2a4847ce48a08a2108ff7fdfd250', 'request_id': 'req-89d45846-5f31-491f-9389-e7c57110a7d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:05.328774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913c2a4847ce48a08a2108ff7fdfd250', 'request_id': 'req-89d45846-5f31-491f-9389-e7c57110a7d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:05.348596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] The token cache is being invalidate because project 913c2a4847ce48a08a2108ff7fdfd250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:09:05.349166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:05.350590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-89d45846-5f31-491f-9389-e7c57110a7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:09:05.353656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 761/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:09:05 2022] DELETE /identity/v3/projects/913c2a4847ce48a08a2108ff7fdfd250 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:09:14.727110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=38 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:09:15.343277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:10:51.676462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e811db-d3a5-4a09-81fc-c422fa0bd186 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:51.677201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e811db-d3a5-4a09-81fc-c422fa0bd186 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:51.677378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e811db-d3a5-4a09-81fc-c422fa0bd186 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:51.679908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:10:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:10:51.685145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86badb49-31a4-4c14-ab75-676c6b77665d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:51.685907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86badb49-31a4-4c14-ab75-676c6b77665d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:51.685907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86badb49-31a4-4c14-ab75-676c6b77665d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:51.706913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-86badb49-31a4-4c14-ab75-676c6b77665d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:10:51.708144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-86badb49-31a4-4c14-ab75-676c6b77665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:10:51.775441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:10:51.950299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e811db-d3a5-4a09-81fc-c422fa0bd186 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:10:51.952514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-70e811db-d3a5-4a09-81fc-c422fa0bd186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:10:52.096090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e811db-d3a5-4a09-81fc-c422fa0bd186 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:10:52.102217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a892d9-edb6-4376-acf1-37921115be42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:10:52.102921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a892d9-edb6-4376-acf1-37921115be42 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:52.103241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a892d9-edb6-4376-acf1-37921115be42 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:52.103660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a892d9-edb6-4376-acf1-37921115be42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:52.114305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a892d9-edb6-4376-acf1-37921115be42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:10:52.116309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a892d9-edb6-4376-acf1-37921115be42 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:10:52.127777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 751/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:10:53.039164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d784cb-61ba-4776-897b-8524cbd01bd4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:53.039752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d784cb-61ba-4776-897b-8524cbd01bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:53.039752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d784cb-61ba-4776-897b-8524cbd01bd4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:53.041174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:10:53.046762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8067c-fe39-41c7-8d8e-911fc9ae597e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:53.047361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8067c-fe39-41c7-8d8e-911fc9ae597e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:53.047361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8067c-fe39-41c7-8d8e-911fc9ae597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:53.070267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-64e8067c-fe39-41c7-8d8e-911fc9ae597e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:10:53.071523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-64e8067c-fe39-41c7-8d8e-911fc9ae597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:10:53.080316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:10:53.237849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d784cb-61ba-4776-897b-8524cbd01bd4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:10:53.254056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d784cb-61ba-4776-897b-8524cbd01bd4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:10:53.260004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c421f49-31e0-4ee6-8961-014a98fdd9e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:10:53.260652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c421f49-31e0-4ee6-8961-014a98fdd9e8 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:53.260928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c421f49-31e0-4ee6-8961-014a98fdd9e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:53.261235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c421f49-31e0-4ee6-8961-014a98fdd9e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:53.271776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c421f49-31e0-4ee6-8961-014a98fdd9e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:10:53.273478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c421f49-31e0-4ee6-8961-014a98fdd9e8 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:10:53.284690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 764/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:10:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:10:54.446756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d87a5-33a7-48b2-8313-0035d07d7922 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:54.447621 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d87a5-33a7-48b2-8313-0035d07d7922 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:54.447621 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d87a5-33a7-48b2-8313-0035d07d7922 None None] PATH_INFO: `/v3` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:54.449160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 753/1517] 10.222.0.52 () {54 vars in 925 bytes} [Tue Jul 26 17:10:54 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:10:54.458287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd05b6-5361-4e21-92b3-d514de0a0797 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:10:54.459161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd05b6-5361-4e21-92b3-d514de0a0797 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:10:54.459358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd05b6-5361-4e21-92b3-d514de0a0797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:10:54.520934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-03cd05b6-5361-4e21-92b3-d514de0a0797 None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:10:54.522203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03cd05b6-5361-4e21-92b3-d514de0a0797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:10:54.625478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 765/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jul 26 17:10:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:11:03.063432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=49 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:11:04.620064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=77 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:11:04.727241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174ff1-0df0-4d50-93ce-b1c185745c9f None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:04.728168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174ff1-0df0-4d50-93ce-b1c185745c9f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:04.728168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed174ff1-0df0-4d50-93ce-b1c185745c9f None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:04.729995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 754/1519] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:11:04.736391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196df725-5d96-4caf-a3f9-db8ab9ea0ef8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:04.736806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196df725-5d96-4caf-a3f9-db8ab9ea0ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:04.737317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196df725-5d96-4caf-a3f9-db8ab9ea0ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:04.762107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-196df725-5d96-4caf-a3f9-db8ab9ea0ef8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:04.763509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-196df725-5d96-4caf-a3f9-db8ab9ea0ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:04.772274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 766/1520] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:11:05.020669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174ff1-0df0-4d50-93ce-b1c185745c9f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:05.037772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed174ff1-0df0-4d50-93ce-b1c185745c9f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:05.043950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20f56a-3c0f-42b4-84b9-2c02610f276d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:05.044583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f56a-3c0f-42b4-84b9-2c02610f276d service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:05.044863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f56a-3c0f-42b4-84b9-2c02610f276d service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:05.045163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20f56a-3c0f-42b4-84b9-2c02610f276d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:05.057323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f56a-3c0f-42b4-84b9-2c02610f276d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:05.060436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20f56a-3c0f-42b4-84b9-2c02610f276d service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:05.079925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 755/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:11:05.964920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70292354-9fa3-4482-a669-31165464a535 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:05.965539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70292354-9fa3-4482-a669-31165464a535 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:05.965806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70292354-9fa3-4482-a669-31165464a535 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:05.967481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 17:11:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:11:05.973784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380969e-aeca-427b-9bbe-d598108d0ad6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:05.974369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380969e-aeca-427b-9bbe-d598108d0ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:05.974472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380969e-aeca-427b-9bbe-d598108d0ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:05.996476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a380969e-aeca-427b-9bbe-d598108d0ad6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:05.997846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a380969e-aeca-427b-9bbe-d598108d0ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:06.007099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 17:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:11:06.111673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55c75-df99-478e-a685-0c849b9a18e3 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:06.112247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55c75-df99-478e-a685-0c849b9a18e3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:06.112537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55c75-df99-478e-a685-0c849b9a18e3 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:06.114043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 768/1524] 10.222.0.48 () {54 vars in 961 bytes} [Tue Jul 26 17:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:11:06.121084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:06.121564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:06.122102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:06.154785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:06.158068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:06.165406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55c75-df99-478e-a685-0c849b9a18e3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:06.173517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 757/1525] 10.222.0.48 () {58 vars in 1058 bytes} [Tue Jul 26 17:11:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:11:06.182239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:06.184327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55c75-df99-478e-a685-0c849b9a18e3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:06.192256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62f28-ca70-4229-b870-0cb128134319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:06.192916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62f28-ca70-4229-b870-0cb128134319 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:06.193233 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62f28-ca70-4229-b870-0cb128134319 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:06.193514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62f28-ca70-4229-b870-0cb128134319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:06.197881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cf46e-1a6b-43a7-b1aa-381d3b7a813a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:06.203358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62f28-ca70-4229-b870-0cb128134319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:06.204822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb072c-645d-4c86-9e42-c9c6ca23069a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:06.205511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb072c-645d-4c86-9e42-c9c6ca23069a service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:06.205701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62f28-ca70-4229-b870-0cb128134319 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:06.205889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb072c-645d-4c86-9e42-c9c6ca23069a service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:06.206064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb072c-645d-4c86-9e42-c9c6ca23069a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:06.215786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb072c-645d-4c86-9e42-c9c6ca23069a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:06.217341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb072c-645d-4c86-9e42-c9c6ca23069a service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:06.218598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 769/1526] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:11:06.229704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 758/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:11:10.185671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da6c7-c491-470d-9a56-e10158e5a1b2 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:10.186384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da6c7-c491-470d-9a56-e10158e5a1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:10.186384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da6c7-c491-470d-9a56-e10158e5a1b2 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:10.187578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:11:10.193572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb43f7-bd8d-4270-bca3-052ba185ea74 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:10.194045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb43f7-bd8d-4270-bca3-052ba185ea74 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:10.194147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb43f7-bd8d-4270-bca3-052ba185ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:10.215175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-eecb43f7-bd8d-4270-bca3-052ba185ea74 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:10.216398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-eecb43f7-bd8d-4270-bca3-052ba185ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:10.225060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:11:10.307450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07da6c7-c491-470d-9a56-e10158e5a1b2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:10.309372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e07da6c7-c491-470d-9a56-e10158e5a1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:10.382444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07da6c7-c491-470d-9a56-e10158e5a1b2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:10.386060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cf31e-7158-423b-94af-d8a985377c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:10.386697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cf31e-7158-423b-94af-d8a985377c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:10.386997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cf31e-7158-423b-94af-d8a985377c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:10.387270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cf31e-7158-423b-94af-d8a985377c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:10.397423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cf31e-7158-423b-94af-d8a985377c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:10.398878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cf31e-7158-423b-94af-d8a985377c90 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:10.409606 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 771/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:11:12.528751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38ba83-cceb-47b7-8bac-924e2ec8f662 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:12.529411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38ba83-cceb-47b7-8bac-924e2ec8f662 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:12.529411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38ba83-cceb-47b7-8bac-924e2ec8f662 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:12.530669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:11:12.537668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31153546-441d-4622-8880-bb85fb337530 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:12.538238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31153546-441d-4622-8880-bb85fb337530 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:12.538681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31153546-441d-4622-8880-bb85fb337530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:12.563373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-31153546-441d-4622-8880-bb85fb337530 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:12.564750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-31153546-441d-4622-8880-bb85fb337530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:12.572849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:11:12.613505 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38ba83-cceb-47b7-8bac-924e2ec8f662 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:12.615275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3d38ba83-cceb-47b7-8bac-924e2ec8f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:12.630544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38ba83-cceb-47b7-8bac-924e2ec8f662 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:12.634459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274fe23-bf58-40ad-a363-790c7a40b91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:12.635124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274fe23-bf58-40ad-a363-790c7a40b91c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:12.635405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274fe23-bf58-40ad-a363-790c7a40b91c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:12.635683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274fe23-bf58-40ad-a363-790c7a40b91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:12.649230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274fe23-bf58-40ad-a363-790c7a40b91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:12.651763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274fe23-bf58-40ad-a363-790c7a40b91c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:12.664608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 761/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:11:16.322279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae571ef-bf01-4e54-ad43-25b97b9463c4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:16.322889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae571ef-bf01-4e54-ad43-25b97b9463c4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:16.322889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae571ef-bf01-4e54-ad43-25b97b9463c4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:16.324420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:11:16.332801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a979f1-1b16-4e6e-9972-b0e5622c0537 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:16.333413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a979f1-1b16-4e6e-9972-b0e5622c0537 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:16.333413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a979f1-1b16-4e6e-9972-b0e5622c0537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:16.355477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-50a979f1-1b16-4e6e-9972-b0e5622c0537 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:16.356695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-50a979f1-1b16-4e6e-9972-b0e5622c0537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:16.366384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:11:16.706598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae571ef-bf01-4e54-ad43-25b97b9463c4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:16.708443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6ae571ef-bf01-4e54-ad43-25b97b9463c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:16.776371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae571ef-bf01-4e54-ad43-25b97b9463c4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:16.780539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9909-ee87-43dd-9d3c-0bc9f1a19e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:16.781218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9909-ee87-43dd-9d3c-0bc9f1a19e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:16.782022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9909-ee87-43dd-9d3c-0bc9f1a19e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:16.782293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9909-ee87-43dd-9d3c-0bc9f1a19e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:16.793610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d9909-ee87-43dd-9d3c-0bc9f1a19e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:16.795078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d9909-ee87-43dd-9d3c-0bc9f1a19e22 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:16.807128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 774/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:11:19.034665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e63c48-772e-4704-9624-0e6ffa4bcaed None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:19.035482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e63c48-772e-4704-9624-0e6ffa4bcaed None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:19.035482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e63c48-772e-4704-9624-0e6ffa4bcaed None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:19.036790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 17:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:11:19.042717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173f9d-0772-4a9d-a984-c649cdb6a4f8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:19.043399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173f9d-0772-4a9d-a984-c649cdb6a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:19.043399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9173f9d-0772-4a9d-a984-c649cdb6a4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:19.065944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d9173f9d-0772-4a9d-a984-c649cdb6a4f8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:11:19.068268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d9173f9d-0772-4a9d-a984-c649cdb6a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:19.076482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 17:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:11:19.411358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e63c48-772e-4704-9624-0e6ffa4bcaed None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:11:19.413415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c7e63c48-772e-4704-9624-0e6ffa4bcaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:11:19.427802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e63c48-772e-4704-9624-0e6ffa4bcaed None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:11:19.431524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c61e7c-ab61-4f04-8b01-13be2c3568b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:11:19.432155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61e7c-ab61-4f04-8b01-13be2c3568b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:11:19.432436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61e7c-ab61-4f04-8b01-13be2c3568b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:11:19.432707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c61e7c-ab61-4f04-8b01-13be2c3568b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:11:19.442645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c61e7c-ab61-4f04-8b01-13be2c3568b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:11:19.444140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c61e7c-ab61-4f04-8b01-13be2c3568b4 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:11:19.455900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 764/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:11:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:11:29.059927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=67 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:11:29.450202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=36 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:53.987262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:53.988226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:53.988226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.017177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:54.018450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:54.031944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 776/1540] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:12:54.050228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.051823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.052710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.099473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:54.102244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:54.125170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 765/1541] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:54.132874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.151747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5505-12c3-4c3b-a5b5-bc8bec7f1552 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.163195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.163853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.164125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.164404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.165169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:54.165615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.167468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.184414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 777/1542] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:12:54.198968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.221515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043033e2-b958-4177-845f-96d1b02fff50 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.234532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2321b2-451e-4feb-808b-2162f9af30fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.236109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2321b2-451e-4feb-808b-2162f9af30fd admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.236888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2321b2-451e-4feb-808b-2162f9af30fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.237752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2321b2-451e-4feb-808b-2162f9af30fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.239259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2321b2-451e-4feb-808b-2162f9af30fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.243401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2321b2-451e-4feb-808b-2162f9af30fd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.318523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 766/1543] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:54.323260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.369534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd00cf-0bd3-4ea3-9f01-735afcddb83a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.512110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f560bf8-5b0e-4467-a5bb-e7138e1897c3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.513001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f560bf8-5b0e-4467-a5bb-e7138e1897c3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.513557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f560bf8-5b0e-4467-a5bb-e7138e1897c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.553086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.553086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.553086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.553086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.554309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.558705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.570189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9f560bf8-5b0e-4467-a5bb-e7138e1897c3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:54.571869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9f560bf8-5b0e-4467-a5bb-e7138e1897c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:54.602892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 767/1544] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:12:54.612497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.613206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.613718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.619537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 778/1545] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:12:54.623058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.654030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c6ea9-2206-401d-8771-12a2bf194be8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.658152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:54.660006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:54.673109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.674166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.674653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.675142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.676607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.680106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.683234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 768/1546] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:54.688509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.707915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f52cb-7d5b-4f72-9eab-8bc031e2e51d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.716719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 779/1547] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:12:54.718648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.729792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.730846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.731355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.731849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.732803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:54.733943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.735433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:12:54.735433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:12:54.735433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:12:54.736770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.752571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9571c4-ff1a-4d65-980f-85e9094cc53f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.758748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 769/1548] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:12:54.768616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.769688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.770197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.770517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.770720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.772312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.775440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.791699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b41fe-3c35-47f8-927c-71e6b5bb791d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.807403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 780/1549] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:54.815196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.817036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.817974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.818486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.818990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.819913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.822180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.854157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af289d-abe4-47e0-b801-408e4de6d912 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:54.874955 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:54.876154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:54.876654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:54.877199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] PATH_INFO: `/v3/projects/46b665a42ad5484daa4dbd039a519696/users/cc57dd5ce53c49999a9847a5240f53e1/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:54.900812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 770/1550] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:54.907928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:54.922639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b665a42ad5484daa4dbd039a519696, user_id=cc57dd5ce53c49999a9847a5240f53e1, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:54.924657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:12:54.924657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:12:54.924657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:12:54.928790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba08c7-5a29-4bb3-8294-255f2067aa5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:54.981913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 781/1551] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:54 2022] PUT /identity/v3/projects/46b665a42ad5484daa4dbd039a519696/users/cc57dd5ce53c49999a9847a5240f53e1/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:55.023158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452a72b-5756-4728-87a7-2ba98d12e340 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.024221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452a72b-5756-4728-87a7-2ba98d12e340 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.024221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452a72b-5756-4728-87a7-2ba98d12e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.059568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e37f12-562d-4ed4-9104-638dba866e32 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.127459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d452a72b-5756-4728-87a7-2ba98d12e340 None None] MFA Rules not processed for user `cc57dd5ce53c49999a9847a5240f53e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.130247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d452a72b-5756-4728-87a7-2ba98d12e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.158701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.159356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.159640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.159912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.160600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.161917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:12:55.161917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:12:55.161917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:12:55.163163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.219185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 771/1552] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:54 2022] POST /identity/v3/users => generated 290 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:55.227268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.253730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cf5b3-961d-4c4f-86d0-56cb9826a6ac admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.272541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2554099-69e2-4573-9548-891634953ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.273526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2554099-69e2-4573-9548-891634953ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.274025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2554099-69e2-4573-9548-891634953ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.274389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2554099-69e2-4573-9548-891634953ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.275420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2554099-69e2-4573-9548-891634953ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.277560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2554099-69e2-4573-9548-891634953ba2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.296157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 772/1553] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:55.297716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a841-9ded-43cc-b2c6-1ccd18a0c3e3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.298012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a841-9ded-43cc-b2c6-1ccd18a0c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.298289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980a841-9ded-43cc-b2c6-1ccd18a0c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.317319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 782/1554] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 26 17:12:55.318823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e9d84-965a-4a8c-a972-cee0a0f1054d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.319144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e9d84-965a-4a8c-a972-cee0a0f1054d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.319424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e9d84-965a-4a8c-a972-cee0a0f1054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.323063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3980a841-9ded-43cc-b2c6-1ccd18a0c3e3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.324330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3980a841-9ded-43cc-b2c6-1ccd18a0c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.334601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 773/1555] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.337726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.338260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.339017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.344141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-574e9d84-965a-4a8c-a972-cee0a0f1054d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.345717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-574e9d84-965a-4a8c-a972-cee0a0f1054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.357103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 783/1556] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.358489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac94ca-7c04-4890-85d5-69dabd8bc3cd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.358939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac94ca-7c04-4890-85d5-69dabd8bc3cd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.359155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac94ca-7c04-4890-85d5-69dabd8bc3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.360724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:55.380134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:55.386228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b1ac94ca-7c04-4890-85d5-69dabd8bc3cd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.388049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b1ac94ca-7c04-4890-85d5-69dabd8bc3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.388526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.390874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.398104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 784/1557] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.399296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e09d0-212e-4fcb-b57e-8b0d41866417 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.399590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e09d0-212e-4fcb-b57e-8b0d41866417 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.399881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e09d0-212e-4fcb-b57e-8b0d41866417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.407779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 774/1558] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.410043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.422898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26c542-6983-4724-8056-f5b31aef4db4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.426132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6a5e09d0-212e-4fcb-b57e-8b0d41866417 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.427413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6a5e09d0-212e-4fcb-b57e-8b0d41866417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.428457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feebf32a-dc79-4fff-8e85-99089ff57f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.429286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf32a-dc79-4fff-8e85-99089ff57f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.429703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf32a-dc79-4fff-8e85-99089ff57f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.430105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebf32a-dc79-4fff-8e85-99089ff57f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.431051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebf32a-dc79-4fff-8e85-99089ff57f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.433130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feebf32a-dc79-4fff-8e85-99089ff57f08 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.436184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 785/1559] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.438122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e01b9a-6c0f-4ac0-b80d-88c49922dd66 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.438415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e01b9a-6c0f-4ac0-b80d-88c49922dd66 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.438697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e01b9a-6c0f-4ac0-b80d-88c49922dd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.460549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 775/1560] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:55.460549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757133d5-77f9-46b3-8b16-373816a125bf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.460549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757133d5-77f9-46b3-8b16-373816a125bf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.460549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757133d5-77f9-46b3-8b16-373816a125bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.461319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-73e01b9a-6c0f-4ac0-b80d-88c49922dd66 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.461797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-73e01b9a-6c0f-4ac0-b80d-88c49922dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.470981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 786/1561] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.472515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5716-da9d-49ba-8180-cef82130595c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.472806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5716-da9d-49ba-8180-cef82130595c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.473120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c5716-da9d-49ba-8180-cef82130595c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.474542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 787/1562] 10.222.0.48 () {54 vars in 962 bytes} [Tue Jul 26 17:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:12:55.479631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19a11-c260-40d8-b59b-a41cb32ff0a6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.480105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19a11-c260-40d8-b59b-a41cb32ff0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.480736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b19a11-c260-40d8-b59b-a41cb32ff0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.499135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-757133d5-77f9-46b3-8b16-373816a125bf None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.501354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-757133d5-77f9-46b3-8b16-373816a125bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.502182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-59b19a11-c260-40d8-b59b-a41cb32ff0a6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.503353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-59b19a11-c260-40d8-b59b-a41cb32ff0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.511507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 788/1563] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.512876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86ac1-564c-495d-bcba-5b29e0bc3d9d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.513195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86ac1-564c-495d-bcba-5b29e0bc3d9d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.513463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d86ac1-564c-495d-bcba-5b29e0bc3d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.518817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 776/1564] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.520735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.521281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.521720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.536818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c7d86ac1-564c-495d-bcba-5b29e0bc3d9d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.537970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c7d86ac1-564c-495d-bcba-5b29e0bc3d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.547647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 789/1565] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.547647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40766890-e03a-410d-8720-e51a4c00284a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.547647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40766890-e03a-410d-8720-e51a4c00284a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.554685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.554858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40766890-e03a-410d-8720-e51a4c00284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.555035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.564794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 777/1566] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.565986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.580628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-40766890-e03a-410d-8720-e51a4c00284a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.581999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-40766890-e03a-410d-8720-e51a4c00284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.582248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4591c-747a-4b76-874e-e583f400540e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.587819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1db379-3ca6-4d23-9f88-750ab9037f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.588763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db379-3ca6-4d23-9f88-750ab9037f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.589217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db379-3ca6-4d23-9f88-750ab9037f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.589756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1db379-3ca6-4d23-9f88-750ab9037f1b admin admin] PATH_INFO: `/v3/projects/25df8851ae9b4039a00a9a30366ac922/users/e7c8648de5544757aa1d91c0878c65d2/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.597161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 790/1567] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.599333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.599950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.600465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.612223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1db379-3ca6-4d23-9f88-750ab9037f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25df8851ae9b4039a00a9a30366ac922, user_id=e7c8648de5544757aa1d91c0878c65d2, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.615242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1db379-3ca6-4d23-9f88-750ab9037f1b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.638710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 778/1568] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:55 2022] PUT /identity/v3/projects/25df8851ae9b4039a00a9a30366ac922/users/e7c8648de5544757aa1d91c0878c65d2/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:55.640426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.640723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.641015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.665348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.665679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.666896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.667119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.740418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 779/1569] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.743420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.756782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacff3a-2d08-495b-be2e-7c173b8019e0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.762316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 791/1570] 10.222.0.48 () {58 vars in 1059 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.762596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.763128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.763317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.763509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.763682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.764423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:55.765716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.767340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.775211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d84453-d881-4c17-b8f1-d0ecfc65233c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.780128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.780818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.781125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.781409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.786159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 780/1571] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:12:55.786403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:55.786403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.786695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.786898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.797854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feb544-a63f-4cb7-bcff-b52e9897e7cd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.798121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 792/1572] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:12:55.798991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.803591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.804237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.804521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.804797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.805555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:55.805881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.807639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.814584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce069-497c-4e6e-a6f5-6e9ad6594355 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.820792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.821740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.822134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.828232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.828448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 781/1573] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:12:55.828448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.828619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:55.828619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.828619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5960-0462-46a6-8d6e-717b245d5a7e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.839120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca895da-6bde-4157-bbd0-a2a0c9a7178d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.844202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.844828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.845152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.845431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.846149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:55.846326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 793/1574] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:12:55.846475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.847535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.847837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.848102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.848300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.866222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 782/1575] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:12:55.867170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.875245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:55.877028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:55.878973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8216f-8cd6-4775-9d0d-dc01d7232a38 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.883718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.884360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.884642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.884918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.889164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 794/1576] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:55.890412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:12:55.890412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.890821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.890989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.904002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f56e-bea9-496f-a6cb-49304473a6f6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.904311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 783/1577] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:12:55.905100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.911007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.911777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.912124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.912468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.917657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94157fd4-4fd0-45c4-8b97-538c4ff42fde admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:55.923336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:55.923975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:55.924269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:55.924536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:55.925258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.927198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.962093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 784/1578] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:55.962093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:55.980293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:55.981967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:55.993154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 795/1579] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:55.994162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.009229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d146a-5226-4c29-96c6-0f4adc53bf32 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.010201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:56.019654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:56.063219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25af8d-c93b-4f8e-bde5-5bfd93426a96 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.071292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.072347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.072710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.073213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.074396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.078538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.088786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.089935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.090480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.091015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.092273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.095063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:12:56.095063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:12:56.095063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:12:56.097507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.127248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 785/1580] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.128370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.143137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 796/1581] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:55 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.145072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.208108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbf463-8e7d-42a3-ac23-b6790b0b0cac admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.254741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46122ea9-9199-4890-bc74-76e095e69e49 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.261857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.262697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.263338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.263829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.265385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.268875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.278607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.279339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.279699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.280043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.281486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.283480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.310864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 797/1582] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.316883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.321687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 786/1583] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.321687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.396617 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:56.401925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926eeb60-4041-4ac5-b651-4731241b09e3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.412014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41911d98-abbd-494e-8ded-4914c04dd81f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.418105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.418978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.419257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.419546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.420236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.422862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.447107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:56.464298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 787/1584] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.465880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.477813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.478928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.479307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.480477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.492366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.494500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.506335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 798/1585] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:56.507259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.520555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee0d7c-d628-483a-8d8b-732a9ff029a1 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.561337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63091555-5af3-45d7-a8b2-97f6ab85c862 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.574883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.575132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.575626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.575845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.576578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.578815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.613714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 788/1586] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.615796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.626851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.628319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.628319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.628582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.629823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.633486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.635857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928554b0-7139-45bf-b033-abea2151802d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.645316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.646482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.647023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.647608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.648747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.652341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.672003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 799/1587] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.674155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.692909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879dbf-cb90-4feb-ad52-ec87b4f7f125 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.693615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 789/1588] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/users => generated 288 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.695101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.706071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c1a68-5297-4567-8fbc-29e29956263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.707156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c1a68-5297-4567-8fbc-29e29956263b admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.707655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c1a68-5297-4567-8fbc-29e29956263b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.708124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c1a68-5297-4567-8fbc-29e29956263b admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.709366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c1a68-5297-4567-8fbc-29e29956263b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.709692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a92ab4-3508-4085-a11b-299bb7789b23 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.713221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c1a68-5297-4567-8fbc-29e29956263b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.715659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.716660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.717225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.718247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.719677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.722120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.752261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 800/1589] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/users => generated 296 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.755136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a95f9-0ec4-4585-b481-df62bec18a25 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.755956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a95f9-0ec4-4585-b481-df62bec18a25 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.756458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a95f9-0ec4-4585-b481-df62bec18a25 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.758535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 790/1590] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/users => generated 287 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.759453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 801/1591] 10.222.0.48 () {54 vars in 912 bytes} [Tue Jul 26 17:12:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:12:56.761104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a95f9-0ec4-4585-b481-df62bec18a25 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.766435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.783727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a95f9-0ec4-4585-b481-df62bec18a25 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.789848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.790576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.790932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.791213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.792081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.794213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.800765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588c913-35ce-4d04-a310-c3753007e1f4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.807048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.807684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.807963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.808233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.808912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.811615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.820731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 802/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:56.821814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.841053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0d0ba-6831-4852-8ca2-3f9a2988b0b7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.853010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 791/1593] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:56.853010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.862462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.863151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.863444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.864063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.864990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.866438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bf78d-81cb-454e-8315-86a6ab3efed5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.866655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.873679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.875166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.875713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.876181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.877546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.880994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.886084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 803/1594] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:56.886084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.907576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 792/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:56.909325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:56.917552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cef79-a87c-4c07-a79c-2287c6278af1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.926256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52bd5d-cfd6-4ca4-9761-6f99e47da9b9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:56.930925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d9f9c-ba76-4aa4-8ac5-d62084ca1a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.932341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d9f9c-ba76-4aa4-8ac5-d62084ca1a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.932901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d9f9c-ba76-4aa4-8ac5-d62084ca1a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.933496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d9f9c-ba76-4aa4-8ac5-d62084ca1a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.934073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:56.934857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d9f9c-ba76-4aa4-8ac5-d62084ca1a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.935087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.935529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.935969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.938330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:56.938621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d9f9c-ba76-4aa4-8ac5-d62084ca1a10 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.941274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:56.969584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 804/1596] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:56.969584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:56.969584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:56.969584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:56.982830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 793/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:56.990787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.011095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d2b73-aaca-45b5-bd3a-c8f8fa1d055f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.012230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:57.014180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:57.024742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.025913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.026426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.026899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.032193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.035638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.071840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 794/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.071840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.090108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46165b-8960-4bc0-bfd4-06255955365d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.098767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.099416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.099770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.100120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.101619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.103355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.116218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 795/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.118840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.139500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705be88-0381-4cd3-bb60-223214f4ed6b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.141610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 805/1600] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:12:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:57.142343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.154130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e87153-f1f6-4a31-9eaf-c209cd4d03c1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.157720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.158470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.158819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.159095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.159976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.161045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.162123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.162279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.162765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.163612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.164594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.166640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.175097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:57.181176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 796/1601] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.184683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.187300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 806/1602] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.188701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.200222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86d0fb-8290-4663-b47a-6ba2a0511c0b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.200523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279f9f0-a036-41a9-b96f-a2b38364e017 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.206087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.206403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.206944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.207118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.207318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.207488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.207701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.207889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.208557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.208752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.210462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.210696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.223874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 807/1603] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.224534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.227606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 797/1604] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.228589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.235039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050e2af-74f9-43e3-a178-18b512e364c8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.241175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf7905-dae2-4918-9802-dc8030a58491 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.241948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.243007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.243250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.243484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] PATH_INFO: `/v3/projects/446eaae0a4f34eba86d4b0de0c2d1013/users/c6291932ceb84b03924c0f056f29c2a9/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.248522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.249879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.250229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.250592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.251474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.253322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.254974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:57.270585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 798/1605] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:57.270585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.279400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=446eaae0a4f34eba86d4b0de0c2d1013, user_id=c6291932ceb84b03924c0f056f29c2a9, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.287579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.308398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 808/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:57 2022] PUT /identity/v3/projects/446eaae0a4f34eba86d4b0de0c2d1013/users/c6291932ceb84b03924c0f056f29c2a9/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:57.309688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.369458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1c6f8-6370-4327-9f5e-685c6d9d0716 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.450074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.450475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185bfdd-af48-4a81-8c38-97de78ffce8a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.450797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.451215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.452678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.458511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.459457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.460013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.460592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] PATH_INFO: `/v3/projects/250e4bed3c4c4c9c94a3143b2c6a7533/users/ab248df5c41d4495b9c9f46ef33c1631/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.482312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250e4bed3c4c4c9c94a3143b2c6a7533, user_id=ab248df5c41d4495b9c9f46ef33c1631, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.486384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.504287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 809/1607] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:57 2022] PUT /identity/v3/projects/250e4bed3c4c4c9c94a3143b2c6a7533/users/ab248df5c41d4495b9c9f46ef33c1631/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:57.505331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.560993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.563605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.578311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 799/1608] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:57.579535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.597733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31100c35-224b-4cf6-b6c6-270c3d4f1f2b service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.608010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.610585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.610936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.611411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] PATH_INFO: `/v3/projects/2b8b6c88af94417c840fdafd1fe431b7/users/97115d07acc5464bb7b097039471fee6/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.621717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fb6e4-ca8e-4ddb-bee2-a3831d054459 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.642970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.644289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8b6c88af94417c840fdafd1fe431b7, user_id=97115d07acc5464bb7b097039471fee6, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.645484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.646389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.647427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] PATH_INFO: `/v3/projects/6f7fbc938d614078b719aa483aa80cd6/users/459db7ff580948dc853b9f843700fe3c/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.647757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.677081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 800/1609] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:57 2022] PUT /identity/v3/projects/2b8b6c88af94417c840fdafd1fe431b7/users/97115d07acc5464bb7b097039471fee6/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:57.678511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.689045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7fbc938d614078b719aa483aa80cd6, user_id=459db7ff580948dc853b9f843700fe3c, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.697275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.730674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 810/1610] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:57 2022] PUT /identity/v3/projects/6f7fbc938d614078b719aa483aa80cd6/users/459db7ff580948dc853b9f843700fe3c/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:57.734169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:57.742220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:57.833194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:57.855208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b71dd5-2105-4f8d-8923-4d083c2da9a9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.903416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629947a-040f-45b7-84d1-f0c6e43d2e0b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:57.911124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c268934-7c0b-49f3-94a2-a820d7b44aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.911841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c268934-7c0b-49f3-94a2-a820d7b44aad admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.912129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c268934-7c0b-49f3-94a2-a820d7b44aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.912463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c268934-7c0b-49f3-94a2-a820d7b44aad admin admin] PATH_INFO: `/v3/projects/64258b46ddf44ea7b47e0e3c002edf0d/users/c6566100cb9c48afa2766d392c7238ef/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.938804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c268934-7c0b-49f3-94a2-a820d7b44aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=64258b46ddf44ea7b47e0e3c002edf0d, user_id=c6566100cb9c48afa2766d392c7238ef, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:57.942769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c268934-7c0b-49f3-94a2-a820d7b44aad admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:57.977000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 811/1611] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:57 2022] PUT /identity/v3/projects/64258b46ddf44ea7b47e0e3c002edf0d/users/c6566100cb9c48afa2766d392c7238ef/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:57.977278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f45db9-9772-4c59-beae-b6073975ce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:57.977426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8068-2138-4049-81e6-c03dde48d2a5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.977426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8068-2138-4049-81e6-c03dde48d2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.977597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f45db9-9772-4c59-beae-b6073975ce84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:57.977674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc8068-2138-4049-81e6-c03dde48d2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.977755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f45db9-9772-4c59-beae-b6073975ce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:57.977755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f45db9-9772-4c59-beae-b6073975ce84 admin admin] PATH_INFO: `/v3/projects/3577acc4ba4c4de2a2503e6ad2163e33/users/d0c9eafcf5894c4aa8b62102c0307490/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:57.993766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f45db9-9772-4c59-beae-b6073975ce84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3577acc4ba4c4de2a2503e6ad2163e33, user_id=d0c9eafcf5894c4aa8b62102c0307490, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:58.001698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f45db9-9772-4c59-beae-b6073975ce84 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:58.005467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-95bc8068-2138-4049-81e6-c03dde48d2a5 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.008360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-95bc8068-2138-4049-81e6-c03dde48d2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.032309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 801/1612] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:57 2022] PUT /identity/v3/projects/3577acc4ba4c4de2a2503e6ad2163e33/users/d0c9eafcf5894c4aa8b62102c0307490/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:58.033958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6ca0-831f-4ed9-87d0-dcdc72c75bbd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.034284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6ca0-831f-4ed9-87d0-dcdc72c75bbd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.034714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac6ca0-831f-4ed9-87d0-dcdc72c75bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.067581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-17ac6ca0-831f-4ed9-87d0-dcdc72c75bbd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.069386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-17ac6ca0-831f-4ed9-87d0-dcdc72c75bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.157845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 802/1613] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:58.159540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ca564-9567-4168-baec-3c193629c068 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.159976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ca564-9567-4168-baec-3c193629c068 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.160351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ca564-9567-4168-baec-3c193629c068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.200025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 812/1614] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:58.202781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a33d-1411-432b-9cdc-384fc4234c04 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.202781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a33d-1411-432b-9cdc-384fc4234c04 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.203116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a33d-1411-432b-9cdc-384fc4234c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.220629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0d3ca564-9567-4168-baec-3c193629c068 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.223380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0d3ca564-9567-4168-baec-3c193629c068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.237488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 803/1615] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:58.237488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.237488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.237488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.274200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cd00a33d-1411-432b-9cdc-384fc4234c04 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.275462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cd00a33d-1411-432b-9cdc-384fc4234c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.282500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] MFA Rules not processed for user `c6566100cb9c48afa2766d392c7238ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.284438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.291712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 813/1616] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:58.295329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.295617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.297708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.334024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] MFA Rules not processed for user `d0c9eafcf5894c4aa8b62102c0307490`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.335282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.404188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:58.436828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 814/1617] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:58.438192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:58.472155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 804/1618] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:58.473428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:58.557460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:58.613355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854ba21-62cd-4df9-8ff2-c01c78f16484 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:58.614611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db23e68-4839-43a4-a179-cfc545983411 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:58.620437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:58.621673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.622340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.622667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.625327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:58.626271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.626711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.627093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.635726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:58.637591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:58.650686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 805/1619] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:58.651484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:58.662699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b13e84-ba21-491b-b1c1-eb3025452abb service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:58.663752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:58.666792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:58.671853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:58.672904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.673335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.674102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.689957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:58.692054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:58.710026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 806/1620] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:58.712194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:58.720228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 815/1621] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:58.721743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:58.725519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af372f-6616-4b42-95ab-7cf23f59b94a service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:58.730921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ff33e-e993-468b-b7cc-b21908d25cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:58.731607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ff33e-e993-468b-b7cc-b21908d25cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.732250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ff33e-e993-468b-b7cc-b21908d25cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.732598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ff33e-e993-468b-b7cc-b21908d25cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.748687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b30bd-a95e-4c41-a048-ce622d26a9fe service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:58.751164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ff33e-e993-468b-b7cc-b21908d25cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:58.753382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ff33e-e993-468b-b7cc-b21908d25cf4 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:58.765781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:58.766775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.767270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.767766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.768828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:58.771921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:58.772040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 807/1622] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:58.774480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ac7dd-59db-4221-9823-bf3aaad149c7 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.774770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ac7dd-59db-4221-9823-bf3aaad149c7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.777857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ac7dd-59db-4221-9823-bf3aaad149c7 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.781264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 808/1623] 10.222.0.48 () {54 vars in 962 bytes} [Tue Jul 26 17:12:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:12:58.787307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:58.787583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:58.787863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:58.857830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:58.860029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:58.869782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 816/1624] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:58.871395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.018090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 809/1625] 10.222.0.48 () {58 vars in 1059 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:59.032542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.054126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3fd8f-66ec-45e6-b576-7426e277d995 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:59.071117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:59.073508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.073807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.074127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.086368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25eaf-b1cf-4850-8640-64c6b0029ebe admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:59.097176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:59.097933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.099452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.099452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.100011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:59.103001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:59.148520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 817/1626] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:12:58 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:12:59.150329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.168689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57b7f1-ac58-486f-a8d4-6d594a3cf854 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:59.177311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:59.178268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.178875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.179376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.181007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:59.185995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:59.217219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 818/1627] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:59.220735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.240589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832afd1-c79f-49d9-af7a-e05d0c50c357 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:59.247168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:59.247969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.248258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.248534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] PATH_INFO: `/v3/projects/e3b2e7c9bee54d1f9d691852431ceced/users/44948de3455f401b978e7e060c30cf05/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.262983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:12:59.279924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2e7c9bee54d1f9d691852431ceced, user_id=44948de3455f401b978e7e060c30cf05, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:59.286650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:59.288707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:59.288830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:59.303801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 810/1628] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:12:59.315706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 819/1629] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:59 2022] PUT /identity/v3/projects/e3b2e7c9bee54d1f9d691852431ceced/users/44948de3455f401b978e7e060c30cf05/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:12:59.319477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.332241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1660822-4371-4a09-b3ca-c344edf09858 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:59.503303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bf89-fe78-44c7-8039-4f966a9b58c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:59.504401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bf89-fe78-44c7-8039-4f966a9b58c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.506035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bf89-fe78-44c7-8039-4f966a9b58c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.506991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bf89-fe78-44c7-8039-4f966a9b58c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.508105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bf89-fe78-44c7-8039-4f966a9b58c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:59.518260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bf89-fe78-44c7-8039-4f966a9b58c0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:59.665764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.666427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 811/1630] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:12:59.666656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81b253-9a7a-4c4d-9c82-8e6cc8ccfaf0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:12:59.666656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:12:59.666656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.666656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.666978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] PATH_INFO: `/v3/projects/e3b2e7c9bee54d1f9d691852431ceced/users/44948de3455f401b978e7e060c30cf05/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.666978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b2e7c9bee54d1f9d691852431ceced, user_id=44948de3455f401b978e7e060c30cf05, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:12:59.666978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:12:59.705222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cfe46-9822-4f3c-a4ac-f2c77ccc9a8f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.705222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cfe46-9822-4f3c-a4ac-f2c77ccc9a8f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.705222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cfe46-9822-4f3c-a4ac-f2c77ccc9a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.721670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 820/1631] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:12:59 2022] PUT /identity/v3/projects/e3b2e7c9bee54d1f9d691852431ceced/users/44948de3455f401b978e7e060c30cf05/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:12:59.755109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3f7cfe46-9822-4f3c-a4ac-f2c77ccc9a8f None None] MFA Rules not processed for user `44948de3455f401b978e7e060c30cf05`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:59.757133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3f7cfe46-9822-4f3c-a4ac-f2c77ccc9a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:12:59.891552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 812/1632] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:12:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:12:59.901485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:12:59.930702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9a9e8-6e08-4034-b251-0a78fdfcc7c5 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.931078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9a9e8-6e08-4034-b251-0a78fdfcc7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.931419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9a9e8-6e08-4034-b251-0a78fdfcc7c5 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.932919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 813/1633] 10.222.0.48 () {54 vars in 912 bytes} [Tue Jul 26 17:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:12:59.938698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:12:59.939311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:12:59.939661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:12:59.985991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:12:59.988897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:00.042011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef362df-3887-4093-8c6f-1166cca2e387 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:00.052872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:00.054068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:00.054343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:00.054783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:00.068180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:00.070599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:00.091482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 821/1634] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:12:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:00.147299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 814/1635] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:12:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:00.183879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:00.194984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:00.209582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793b295-ccf5-44c7-b573-e98cb63d7767 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:00.216725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:00.224663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:00.225662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:00.225662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:00.287333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:00.290849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:00.314022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 822/1636] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:00.638655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:00.641787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:00.651303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:00.691726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5def9-7edc-4a00-bc82-9d72b22128b4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:00.709724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e03b4-ae52-4cd3-ae79-df845fb02129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:00.713020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e03b4-ae52-4cd3-ae79-df845fb02129 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:00.714496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e03b4-ae52-4cd3-ae79-df845fb02129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:00.716612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e03b4-ae52-4cd3-ae79-df845fb02129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:00.822030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daff52-17d1-4d15-aab8-e2ab8764dea9 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:00.833768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:00.835464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:00.836139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:00.836919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] PATH_INFO: `/v3/users/d0c9eafcf5894c4aa8b62102c0307490` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:00.839225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c9eafcf5894c4aa8b62102c0307490)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:00.842374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: WARNING py.warnings [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:13:00.842374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: warnings.warn(msg) Jul 26 17:13:00.842374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: Jul 26 17:13:00.885204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:00.959150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e03b4-ae52-4cd3-ae79-df845fb02129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:00.961443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e03b4-ae52-4cd3-ae79-df845fb02129 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:00.974308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 815/1637] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:01.105101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c9eafcf5894c4aa8b62102c0307490'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:01.146121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c9eafcf5894c4aa8b62102c0307490'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:01.160192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c9eafcf5894c4aa8b62102c0307490'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:01.160811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c9eafcf5894c4aa8b62102c0307490'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:01.161980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c9eafcf5894c4aa8b62102c0307490'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:01.214605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 823/1638] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:00 2022] DELETE /identity/v3/users/d0c9eafcf5894c4aa8b62102c0307490 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:01.222360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:01.223078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:01.223667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:01.258031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:01.260994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:01.386004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 816/1639] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:01.466374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:01.468038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:01.633289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb56602-84b4-49ce-8223-8fd3f37940a1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:01.637861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb883b-0d2b-4b5c-b3c3-4b7d5d1ae55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:01.638867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb883b-0d2b-4b5c-b3c3-4b7d5d1ae55c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:01.639954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb883b-0d2b-4b5c-b3c3-4b7d5d1ae55c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:01.640661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb883b-0d2b-4b5c-b3c3-4b7d5d1ae55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:01.652992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb883b-0d2b-4b5c-b3c3-4b7d5d1ae55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:01.655477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb883b-0d2b-4b5c-b3c3-4b7d5d1ae55c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:01.684137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 824/1640] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:04.742221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:04.746354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:04.807250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9cbba-bfa5-470b-ba18-4cb80baf5a6a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:04.833409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:04.834479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:04.835108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:04.835868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] PATH_INFO: `/v3/projects/3577acc4ba4c4de2a2503e6ad2163e33` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:04.840901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3577acc4ba4c4de2a2503e6ad2163e33)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:04.847798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: WARNING py.warnings [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 17:13:04.847798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: warnings.warn(msg) Jul 26 17:13:04.847798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: Jul 26 17:13:04.855256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:05.008526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3577acc4ba4c4de2a2503e6ad2163e33', 'request_id': 'req-199f44bf-f489-496c-977d-3014f074950c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:05.026279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3577acc4ba4c4de2a2503e6ad2163e33', 'request_id': 'req-199f44bf-f489-496c-977d-3014f074950c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:05.066118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3577acc4ba4c4de2a2503e6ad2163e33', 'request_id': 'req-199f44bf-f489-496c-977d-3014f074950c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:05.104595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] The token cache is being invalidate because project 3577acc4ba4c4de2a2503e6ad2163e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:05.104772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:05.105548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-199f44bf-f489-496c-977d-3014f074950c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:05.111987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 817/1641] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:04 2022] DELETE /identity/v3/projects/3577acc4ba4c4de2a2503e6ad2163e33 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:05.114034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.114666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.114666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.175285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:05.176561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:05.207064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:05.377822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 825/1642] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:05.384044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.384433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.384781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.436270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:05.438882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:05.454766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 818/1643] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:05.458307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:05.477440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954631b-708e-4e00-b3d6-a23ae6fcc756 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:05.490642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:05.491180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.491466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.492106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.492484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:05.492779 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:05.494717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:05.527023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 826/1644] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:05.533090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:05.548607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577acc7c-5504-4d8e-ac96-4024a7d1b535 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:05.554795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:05.555634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.556063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.556627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.557414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:05.560008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:05.581439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:05.614056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:05.614795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 819/1645] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:05 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:05.631513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81764c9-f7a4-4e0d-ac03-4fc1526719b3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:05.807007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:05.807675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.808112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.808402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.809128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:05.811164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:05.839608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 827/1646] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:05 2022] POST /identity/v3/users => generated 291 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:05.843827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:05.863280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94caae-9abf-4dcb-9625-312ff3b39e88 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:05.874804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:05.875631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.875985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.876335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.877319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:05.880134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:05.900380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 820/1647] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:13:05.901694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:05.916810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56701f0-4cdd-4e8b-a0c7-dbf62cb5087e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:05.927637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea87952-c41e-4a0b-a587-6fb1027c539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:05.928497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea87952-c41e-4a0b-a587-6fb1027c539c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:05.928878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea87952-c41e-4a0b-a587-6fb1027c539c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:05.929988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea87952-c41e-4a0b-a587-6fb1027c539c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:05.930886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea87952-c41e-4a0b-a587-6fb1027c539c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:05.932831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea87952-c41e-4a0b-a587-6fb1027c539c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:05.952688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 828/1648] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:05.957291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:05.999162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a9b5f-dce0-4f70-bd3e-f42cce1acca0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:06.007516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:06.008791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:06.009383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:06.009768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] PATH_INFO: `/v3/projects/b66d9eef7815468cb7ba69b281ddb0e1/users/5b04e3a585e04c9abaf7037febff124b/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:06.049623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66d9eef7815468cb7ba69b281ddb0e1, user_id=5b04e3a585e04c9abaf7037febff124b, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:06.056192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:06.088641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 821/1649] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:05 2022] PUT /identity/v3/projects/b66d9eef7815468cb7ba69b281ddb0e1/users/5b04e3a585e04c9abaf7037febff124b/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:06.109419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:06.110445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:06.110854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:06.150342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] MFA Rules not processed for user `5b04e3a585e04c9abaf7037febff124b`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:06.152979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:06.304283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 829/1650] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:06.310902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:06.660745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e95aca-3ade-4e24-86b3-ce3eddee80cd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:06.673848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:06.673848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:06.673848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:06.673848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:06.674535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:06.680243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:06.703462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:06.712660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:06.770586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fcb04-2bbd-47ea-a8e3-3aba623038a5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:06.779067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:06.783151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:06.783445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:06.787395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 822/1651] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:06 2022] POST /identity/v3/projects => generated 381 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:06.788995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] PATH_INFO: `/v3/users/c6566100cb9c48afa2766d392c7238ef` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:06.791958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6566100cb9c48afa2766d392c7238ef)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:06.792995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:06.800202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:06.802861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:06.968664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e86a69-59a1-4d49-b8c6-dfea75bdbec4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:06.998320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6566100cb9c48afa2766d392c7238ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:06.999098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176e837-0048-423f-810b-312e492943bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:07.000615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176e837-0048-423f-810b-312e492943bf admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.001078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176e837-0048-423f-810b-312e492943bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.001520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176e837-0048-423f-810b-312e492943bf admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.003349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176e837-0048-423f-810b-312e492943bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:07.006328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176e837-0048-423f-810b-312e492943bf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:07.026975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6566100cb9c48afa2766d392c7238ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.062773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6566100cb9c48afa2766d392c7238ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.063665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6566100cb9c48afa2766d392c7238ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.064571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a90d0d79-2c7d-4272-afd3-eb7f21d6a052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6566100cb9c48afa2766d392c7238ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.078238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 823/1652] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:06 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:07.082366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176e837-0048-423f-810b-312e492943bf admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:07.089274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6176e837-0048-423f-810b-312e492943bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:07.131017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 830/1653] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:06 2022] DELETE /identity/v3/users/c6566100cb9c48afa2766d392c7238ef => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:07.142805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3298f2-f125-46c9-a066-eb1e194432ea None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.143271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3298f2-f125-46c9-a066-eb1e194432ea None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.145438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3298f2-f125-46c9-a066-eb1e194432ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.152424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176e837-0048-423f-810b-312e492943bf admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:07.220438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6c3298f2-f125-46c9-a066-eb1e194432ea None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:07.222383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6c3298f2-f125-46c9-a066-eb1e194432ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:07.295228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:07.415101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:07.416338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.417495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.418600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.420055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:07.424706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:07.434475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 831/1654] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:07.435776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11b379-b776-496c-948e-7300a02e869f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.436070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11b379-b776-496c-948e-7300a02e869f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.436482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11b379-b776-496c-948e-7300a02e869f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.475776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 824/1655] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:07.478929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:07.485637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:07.499053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2d11b379-b776-496c-948e-7300a02e869f None None] MFA Rules not processed for user `e7c8648de5544757aa1d91c0878c65d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:07.509242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2d11b379-b776-496c-948e-7300a02e869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:07.708181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390821c-77cc-44d1-90c5-8a27846c7a2b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:07.716865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:07.719743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.720178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.720641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.726875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 832/1656] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:07.727990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11b379-b776-496c-948e-7300a02e869f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:07.742340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2d11b379-b776-496c-948e-7300a02e869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:07.750325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:07.752675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:07.774103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11b379-b776-496c-948e-7300a02e869f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:07.781577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:07.782982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.784051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.784343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] PATH_INFO: `/v3/users/cc57dd5ce53c49999a9847a5240f53e1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.785827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc57dd5ce53c49999a9847a5240f53e1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:07.787965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:07.792036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:07.816432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 825/1657] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:07.822539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:07.859980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdc94c-da38-4870-9348-e3de50a6f93c service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:07.877923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951371cd-d257-4e1b-9c5a-86410cee69a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:07.881470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951371cd-d257-4e1b-9c5a-86410cee69a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.882092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951371cd-d257-4e1b-9c5a-86410cee69a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.882678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951371cd-d257-4e1b-9c5a-86410cee69a5 admin admin] PATH_INFO: `/v3/projects/7c611ec0796041b49ce93bbc7f3cef37/users/597102f2fe924c07a1ef5ecba008ef5d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.882965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc57dd5ce53c49999a9847a5240f53e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.905110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc57dd5ce53c49999a9847a5240f53e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.915640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc57dd5ce53c49999a9847a5240f53e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.916641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc57dd5ce53c49999a9847a5240f53e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.917303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9e4036bf-4b3e-49b0-925e-dbd0a67dc16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc57dd5ce53c49999a9847a5240f53e1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:07.935202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951371cd-d257-4e1b-9c5a-86410cee69a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c611ec0796041b49ce93bbc7f3cef37, user_id=597102f2fe924c07a1ef5ecba008ef5d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:07.944524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951371cd-d257-4e1b-9c5a-86410cee69a5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:07.945806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 833/1658] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:07 2022] DELETE /identity/v3/users/cc57dd5ce53c49999a9847a5240f53e1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:07.947516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.947817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.948106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:07.975708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] MFA Rules not processed for user `e7c8648de5544757aa1d91c0878c65d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:07.977676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:07.992822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 826/1659] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:07 2022] PUT /identity/v3/projects/7c611ec0796041b49ce93bbc7f3cef37/users/597102f2fe924c07a1ef5ecba008ef5d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:07.996376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:07.997547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:07.998101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:08.063560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:08.069146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.089404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 834/1660] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:08.091072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:08.103727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.242734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 827/1661] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:08.244282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884e3e6-8b2a-4251-9b65-acd4272c69ed None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:08.244722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:08.248087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.260129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:08.267404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:08.268345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:08.269293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:08.271141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:08.275046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:08.301500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 835/1662] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:08.306995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:08.334442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbecbb-c99f-4d9d-ae37-44a53e3a5148 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:08.358016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940bf7-7ba8-4eeb-8e86-125072a2422d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:08.358951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940bf7-7ba8-4eeb-8e86-125072a2422d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:08.359447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940bf7-7ba8-4eeb-8e86-125072a2422d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:08.359815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940bf7-7ba8-4eeb-8e86-125072a2422d admin admin] PATH_INFO: `/v3/projects/7c611ec0796041b49ce93bbc7f3cef37/users/597102f2fe924c07a1ef5ecba008ef5d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:08.365447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940bf7-7ba8-4eeb-8e86-125072a2422d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c611ec0796041b49ce93bbc7f3cef37, user_id=597102f2fe924c07a1ef5ecba008ef5d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:08.371237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940bf7-7ba8-4eeb-8e86-125072a2422d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:08.403932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:08.416244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 836/1663] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:08 2022] PUT /identity/v3/projects/7c611ec0796041b49ce93bbc7f3cef37/users/597102f2fe924c07a1ef5ecba008ef5d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:08.429539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:08.430276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:08.430767 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:08.451076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9e44a-3b0d-4a37-8401-42d6906a1686 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:08.494194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:08.529545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] MFA Rules not processed for user `597102f2fe924c07a1ef5ecba008ef5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:08.531801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.589112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:08.590487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:08.590944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:08.591656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:08.594722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.694458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 837/1664] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:08.695401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:08.697879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.705886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:08.710328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:08.755787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 828/1665] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:08.757773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:08.763002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:08.870918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5055c-b735-4a00-aaa0-6f5c4071627d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:08.880545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:08.885702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:08.885702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:08.886232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:08.948474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76592334-8bc2-4c80-b809-d65150380e1b service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:08.965586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:08.972356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:08.972901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:08.973427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:09.024457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:09.027473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:09.082283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 829/1666] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:09.183806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:09.187235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:09.210145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 838/1667] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:09.343817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:09.390797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d1ac6-a974-47e9-beae-0930a2cf33c9 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:09.437049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:09.440724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:09.441466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:09.442485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:09.445844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:09.513669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:09.560918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:09.564032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938a2b6-1a06-4b96-bba1-10a3887ed7f2 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:09.599911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 830/1668] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:10.897317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:10.903133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:10.962801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3b51b-7d22-4f8b-9799-533ebaff7c90 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:10.973747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:10.976283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:10.976816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:10.977396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] PATH_INFO: `/v3/projects/46b665a42ad5484daa4dbd039a519696` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:10.979731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b665a42ad5484daa4dbd039a519696)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:10.984647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:11.134275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:11.157638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b665a42ad5484daa4dbd039a519696', 'request_id': 'req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.160867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59800e-5e85-41ae-92b5-f362b9eeef3a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:11.161450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59800e-5e85-41ae-92b5-f362b9eeef3a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:11.161529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59800e-5e85-41ae-92b5-f362b9eeef3a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:11.163465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 831/1669] 10.222.0.48 () {54 vars in 912 bytes} [Tue Jul 26 17:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:13:11.172713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:11.173094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b665a42ad5484daa4dbd039a519696', 'request_id': 'req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.173194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:11.173396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:11.195136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b665a42ad5484daa4dbd039a519696', 'request_id': 'req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.239661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] The token cache is being invalidate because project 46b665a42ad5484daa4dbd039a519696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:11.239963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.240670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc867bc3-b9b8-44b9-9cf4-9e0aa4d33d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.249609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 839/1670] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:10 2022] DELETE /identity/v3/projects/46b665a42ad5484daa4dbd039a519696 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:11.256240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f000d-a81e-478a-a26d-0f05729c51ad None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:11.256562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f000d-a81e-478a-a26d-0f05729c51ad None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:11.256815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f000d-a81e-478a-a26d-0f05729c51ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:11.314938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-594f000d-a81e-478a-a26d-0f05729c51ad None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:11.316628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-594f000d-a81e-478a-a26d-0f05729c51ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.334199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:11.341281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.562603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 832/1671] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:13:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:11.562990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 840/1672] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:11.564295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:11.570562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:11.570652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:11.571136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:11.579157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.603108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9481b20-577c-40de-a3e6-a986f1c6c4dd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:11.617600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:11.618962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:11.619615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:11.620107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] PATH_INFO: `/v3/projects/64258b46ddf44ea7b47e0e3c002edf0d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:11.622483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64258b46ddf44ea7b47e0e3c002edf0d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:11.628418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:11.678929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:11.681022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.729536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 841/1673] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:11.741436 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:11.746872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.784932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64258b46ddf44ea7b47e0e3c002edf0d', 'request_id': 'req-ba6809bf-48f3-493b-b25b-765066f7ce02'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.795400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d1619-999c-4955-8263-a194ee486ab1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:11.809665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64258b46ddf44ea7b47e0e3c002edf0d', 'request_id': 'req-ba6809bf-48f3-493b-b25b-765066f7ce02'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.831632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64258b46ddf44ea7b47e0e3c002edf0d', 'request_id': 'req-ba6809bf-48f3-493b-b25b-765066f7ce02'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.883272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] The token cache is being invalidate because project 64258b46ddf44ea7b47e0e3c002edf0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:11.883571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.890417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:11.893829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:11.894762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:11.895257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:11.895724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:11.899243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.901212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 833/1674] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:11 2022] DELETE /identity/v3/projects/64258b46ddf44ea7b47e0e3c002edf0d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:11.903149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:11.906994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:11.934418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:11.939027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:11.965521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 842/1675] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:11.966464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:11.972035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:12.076620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:12.139313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba8a21-8f2e-4d66-af1c-aa7a57de2b57 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:12.143578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6809bf-48f3-493b-b25b-765066f7ce02 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:12.153099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:12.157587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:12.157910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:12.160750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] PATH_INFO: `/v3/users/44948de3455f401b978e7e060c30cf05` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:12.162130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:12.163547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:12.164047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:12.164528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:12.165540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:12.166773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] RBAC: Authorizing `identity:delete_user(user_id=44948de3455f401b978e7e060c30cf05)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:12.167432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:12.174299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:12.185596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:12.195600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:12.256939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 834/1676] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:12.259850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:12.298415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d49979-65c2-4bff-8f13-11c6978b9430 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:12.301882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44948de3455f401b978e7e060c30cf05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:12.309444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:12.310907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:12.315933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:12.317453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:12.318407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:12.321092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:12.323215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44948de3455f401b978e7e060c30cf05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:12.339417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44948de3455f401b978e7e060c30cf05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:12.340063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44948de3455f401b978e7e060c30cf05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:12.340757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44948de3455f401b978e7e060c30cf05'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:12.382136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 843/1677] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:11 2022] DELETE /identity/v3/users/44948de3455f401b978e7e060c30cf05 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:12.426238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 835/1678] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:12 2022] POST /identity/v3/projects => generated 405 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:12.427433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:12.428878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:12.552453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad326bb-e9fc-4a67-9659-2e91e206378c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:12.687474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:12.688478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:12.688938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:12.689423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:12.690307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:12.692736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:12.741477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 844/1679] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:12 2022] POST /identity/v3/users => generated 303 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:12.746112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:12.768490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a978be-8c7f-424d-a500-990fecbfc320 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:12.787369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:12.787369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:12.787369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:12.787369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:12.788035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:12.790528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:12.826623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 836/1680] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:12.832198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:12.844379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2db3-73dc-4291-958f-75807cb35997 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:12.876421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312e14e-eb21-41d8-b161-3fc44483ae9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:12.877438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e14e-eb21-41d8-b161-3fc44483ae9f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:12.877908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e14e-eb21-41d8-b161-3fc44483ae9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:12.878418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e14e-eb21-41d8-b161-3fc44483ae9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:12.879241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312e14e-eb21-41d8-b161-3fc44483ae9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:12.885532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312e14e-eb21-41d8-b161-3fc44483ae9f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:12.938883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 845/1681] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:12.942320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:12.979503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7b40-f73b-48b6-9a42-e9912be40142 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:13.002726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb926d-2b81-443a-8acc-c16957cff546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:13.003433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb926d-2b81-443a-8acc-c16957cff546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:13.003719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb926d-2b81-443a-8acc-c16957cff546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:13.004319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb926d-2b81-443a-8acc-c16957cff546 admin admin] PATH_INFO: `/v3/projects/033dca51365847a3b3cdea0018799a78/users/051506e3708249738a7862b4dbd0eb6d/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:13.053134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb926d-2b81-443a-8acc-c16957cff546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033dca51365847a3b3cdea0018799a78, user_id=051506e3708249738a7862b4dbd0eb6d, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:13.056249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb926d-2b81-443a-8acc-c16957cff546 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:13.093237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 837/1682] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:12 2022] PUT /identity/v3/projects/033dca51365847a3b3cdea0018799a78/users/051506e3708249738a7862b4dbd0eb6d/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:13.098861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:13.099159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:13.099437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:13.162922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:13.169251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:13.188266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:13.393092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df8ff-4c86-4e04-96c7-8f4d0135077a None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:13.394194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df8ff-4c86-4e04-96c7-8f4d0135077a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:13.394779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810df8ff-4c86-4e04-96c7-8f4d0135077a None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:13.396894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 838/1683] 10.222.0.48 () {54 vars in 961 bytes} [Tue Jul 26 17:13:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:13:13.413922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:13.414481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:13.415229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:13.423134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 846/1684] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:13.440904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:13.443364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:13.512934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:13.515699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:13.627541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 839/1685] 10.222.0.48 () {58 vars in 1058 bytes} [Tue Jul 26 17:13:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:13.629854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:13.644437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f28f5-7252-45bc-ba08-f01f0a06c87f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:13.651449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:13.656011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:13.656603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:13.657228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:13.662453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea63521-5bd5-4de5-a500-e4770c9e7ae3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:13.663690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:13.673758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42a4f3-34bb-45e9-b395-b1b1191a856c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:13.676972 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a4f3-34bb-45e9-b395-b1b1191a856c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:13.679176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a4f3-34bb-45e9-b395-b1b1191a856c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:13.679280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42a4f3-34bb-45e9-b395-b1b1191a856c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:13.691507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:13.695971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:13.715273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42a4f3-34bb-45e9-b395-b1b1191a856c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:13.718174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42a4f3-34bb-45e9-b395-b1b1191a856c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:13.737462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 840/1686] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:13.754938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 847/1687] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:14.969812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:14.971724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:15.002324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2382f45-37a3-4c3a-94e9-46d3e6a3e6b9 service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:15.023462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:15.024470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:15.025037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:15.025570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] PATH_INFO: `/v3/projects/e3b2e7c9bee54d1f9d691852431ceced` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:15.029533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b2e7c9bee54d1f9d691852431ceced)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:15.035753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:15.202961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b2e7c9bee54d1f9d691852431ceced', 'request_id': 'req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:15.232212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b2e7c9bee54d1f9d691852431ceced', 'request_id': 'req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:15.262298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b2e7c9bee54d1f9d691852431ceced', 'request_id': 'req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:15.264313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:15.300728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] The token cache is being invalidate because project e3b2e7c9bee54d1f9d691852431ceced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:15.301287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:15.309080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-239b7caa-97d7-4c0f-95b3-3c0b7b9a0238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:15.316745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 841/1688] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:14 2022] DELETE /identity/v3/projects/e3b2e7c9bee54d1f9d691852431ceced => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:15.318304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:15.318808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:15.318896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:15.380754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:15.382054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:15.604990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:15.606940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 848/1689] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:15.607315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:15.607686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:15.651964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:15.653347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:15.684318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 842/1690] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:15.685656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:15.699879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57a07a-5e6d-4594-afa3-528d1347a9d3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:15.724430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:15.725269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:15.725591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:15.725873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:15.726607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:15.727094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:15.728576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:15.775682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 849/1691] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:15.777628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:15.808236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a881086-dcf5-4d93-b8d9-80ce530c7aab None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:15.829923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:15.830646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:15.831024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:15.831398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:15.832180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:15.834859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:15.904888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 843/1692] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:15 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:15.910208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:15.925026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fe325-5e39-4375-acf5-9b6620850c33 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:16.028294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:16.073360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:16.074070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.074335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.074608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.075324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:16.078500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:16.127631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 850/1693] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:15 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:16.132280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:16.152284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb6a34-ca22-4bd4-ac09-862ab7cf35d4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:16.168018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451c5c4-c15a-4cef-a319-e3fbd0076c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:16.169332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451c5c4-c15a-4cef-a319-e3fbd0076c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.169853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451c5c4-c15a-4cef-a319-e3fbd0076c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.170327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451c5c4-c15a-4cef-a319-e3fbd0076c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.171321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451c5c4-c15a-4cef-a319-e3fbd0076c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:16.175264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451c5c4-c15a-4cef-a319-e3fbd0076c85 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:16.208353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 844/1694] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:16.212812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:16.226659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.227494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.227999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.231348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d925e78-4826-4a49-bd85-65e70ad9e08a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:16.253761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:16.254436 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.254714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.254986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.255695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:16.265859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:16.298343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] MFA Rules not processed for user `c6291932ceb84b03924c0f056f29c2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:16.299874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:16.303957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 851/1695] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:16.308068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:16.345434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75e3e4-2e46-480e-a10b-1aab837ccc41 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:16.357155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118646e3-7622-4817-940c-de99ab430c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:16.357851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118646e3-7622-4817-940c-de99ab430c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.359097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118646e3-7622-4817-940c-de99ab430c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.359386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118646e3-7622-4817-940c-de99ab430c95 admin admin] PATH_INFO: `/v3/projects/f2122cd66f434c50a00cd63b893c9b9a/users/6899853f27d342b29c44e0fea9087fae/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.387909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118646e3-7622-4817-940c-de99ab430c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2122cd66f434c50a00cd63b893c9b9a, user_id=6899853f27d342b29c44e0fea9087fae, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:16.393167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118646e3-7622-4817-940c-de99ab430c95 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:16.432906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 852/1696] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:16 2022] PUT /identity/v3/projects/f2122cd66f434c50a00cd63b893c9b9a/users/6899853f27d342b29c44e0fea9087fae/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:16.434673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.434961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.435236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.436666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 845/1697] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:16.457467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:16.459471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:16.490740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] MFA Rules not processed for user `6899853f27d342b29c44e0fea9087fae`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:16.492831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:16.719767 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 853/1698] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:16.721952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:16.757867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2616296-ebeb-40e5-a831-d7765c554182 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:16.848714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:16.918875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8d82-3032-428a-840f-5fa5ff95e829 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:16.940699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:16.942706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.946228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.946526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:16.947393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:16.952234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:16.969909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:16.970638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:16.971271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:16.971271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:17.085054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 854/1699] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:16 2022] POST /identity/v3/projects => generated 381 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:17.088689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:17.115369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:17.194425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:17.196076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:17.230407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 846/1700] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:17.378573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4edc182-20ac-4a2f-946f-98cfb3e9459c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:17.406528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695d9c7-5583-48b8-9c96-06efab39ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:17.407649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d9c7-5583-48b8-9c96-06efab39ed90 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:17.407934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d9c7-5583-48b8-9c96-06efab39ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:17.408276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695d9c7-5583-48b8-9c96-06efab39ed90 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:17.408901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695d9c7-5583-48b8-9c96-06efab39ed90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:17.426683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695d9c7-5583-48b8-9c96-06efab39ed90 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:17.514028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 855/1701] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:17 2022] POST /identity/v3/users => generated 297 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:17.514127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:17.526202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b243a8-3e55-4b62-a174-6691ab0d6981 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:17.550334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:17.551852 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:17.553789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:17.555466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:17.558788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:17.560628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:17.589918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:17.590251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:17.590626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:17.608838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 847/1702] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:17.609994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:17.660148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c2669-c4ab-4508-b4a5-2b03e0b5f07d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:17.681164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:17.681877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:17.682163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:17.682434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] PATH_INFO: `/v3/projects/4aad38a85bad4dbea4d518323eab5102/users/1d0d5c0141354c66bf93e05c62b55bcf/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:17.708493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] MFA Rules not processed for user `051506e3708249738a7862b4dbd0eb6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:17.711828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:17.762029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aad38a85bad4dbea4d518323eab5102, user_id=1d0d5c0141354c66bf93e05c62b55bcf, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:17.765818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:17.819470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 848/1703] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:17 2022] PUT /identity/v3/projects/4aad38a85bad4dbea4d518323eab5102/users/1d0d5c0141354c66bf93e05c62b55bcf/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:17.839024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:17.942620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 856/1704] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:17.944122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:17.950061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:18.009311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e99d83-5432-486e-b4f3-9d325d0e767b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:18.055073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf92d06-6ff9-4627-98ef-1504d1573465 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:18.067216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:18.067872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:18.068150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:18.068420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:18.069326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:18.071194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:18.100021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 849/1705] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:18.104079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:18.128837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa6bff-409e-43da-a4c6-a3e0687317f0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:18.146775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67cd7c0-48af-435e-8fcb-6d0471c447b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:18.147666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cd7c0-48af-435e-8fcb-6d0471c447b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:18.148121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cd7c0-48af-435e-8fcb-6d0471c447b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:18.148550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67cd7c0-48af-435e-8fcb-6d0471c447b6 admin admin] PATH_INFO: `/v3/projects/4aad38a85bad4dbea4d518323eab5102/users/1d0d5c0141354c66bf93e05c62b55bcf/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:18.173649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cd7c0-48af-435e-8fcb-6d0471c447b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aad38a85bad4dbea4d518323eab5102, user_id=1d0d5c0141354c66bf93e05c62b55bcf, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:18.187841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67cd7c0-48af-435e-8fcb-6d0471c447b6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:18.240814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:18.241832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:18.246063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 850/1706] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:18 2022] PUT /identity/v3/projects/4aad38a85bad4dbea4d518323eab5102/users/1d0d5c0141354c66bf93e05c62b55bcf/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:18.249641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:18.249992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:18.250737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:18.253377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:18.272441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:18.272994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:18.273422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:18.333706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 857/1707] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:17 2022] POST /identity/v3/projects => generated 405 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:18.338831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:18.411312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] MFA Rules not processed for user `1d0d5c0141354c66bf93e05c62b55bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:18.419285 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:18.430539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:18.496776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:18.798890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f7771-5e5b-4720-b136-333e0e25eba1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:18.807875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:18.808742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:18.808928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:18.809347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:18.814947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:18.818589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:18.884506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 851/1708] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 607 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:18.893172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 858/1709] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:18 2022] POST /identity/v3/users => generated 309 bytes in 556 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:18.897929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:18.906755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:18.914406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:18.935700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26404abf-fe4e-42f0-82b3-163383612599 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:18.959972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:18.960940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:18.962731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:18.963201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:18.966118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:18.972551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:19.019502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 852/1710] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:19.027121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:19.066399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dbdf2-e811-43f4-a410-696083b11137 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:19.076725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:19.083598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:19.085156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:19.087881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] PATH_INFO: `/v3/projects/7200df1c877e407b88af53b2f32149bd/users/80daf046f3d34357b513374573f17791/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:19.127503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7200df1c877e407b88af53b2f32149bd, user_id=80daf046f3d34357b513374573f17791, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:19.131931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:19.177443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 853/1711] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:19 2022] PUT /identity/v3/projects/7200df1c877e407b88af53b2f32149bd/users/80daf046f3d34357b513374573f17791/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:19.185640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:19.255463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b365541-5f5d-4931-a606-510e87baa97c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:19.470337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:19.470991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:19.472872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:19.473193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:19.694699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f2044-ffc1-418b-9801-418a01b6d499 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:19.707034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace87b3-b55b-4bbb-bdc6-fb82eb13b841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:19.707772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace87b3-b55b-4bbb-bdc6-fb82eb13b841 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:19.708126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace87b3-b55b-4bbb-bdc6-fb82eb13b841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:19.708491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace87b3-b55b-4bbb-bdc6-fb82eb13b841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:19.709366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace87b3-b55b-4bbb-bdc6-fb82eb13b841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:19.712076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace87b3-b55b-4bbb-bdc6-fb82eb13b841 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:19.742722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 854/1712] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 558 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:19.751708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:19.751946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:19.752252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:19.823252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:19.824812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:19.907784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 859/1713] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 1003 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:19.914469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:19.918001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:19.921718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:19.982506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20650145-36e7-49f1-ae28-8eed0b11648d service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:20.012036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:20.013334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:20.016320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:20.016675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] PATH_INFO: `/v3/projects/7200df1c877e407b88af53b2f32149bd/users/80daf046f3d34357b513374573f17791/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:20.024026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7200df1c877e407b88af53b2f32149bd, user_id=80daf046f3d34357b513374573f17791, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:20.037390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:20.046853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:20.080268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 860/1714] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:19 2022] PUT /identity/v3/projects/7200df1c877e407b88af53b2f32149bd/users/80daf046f3d34357b513374573f17791/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:20.092149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:20.162726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 855/1715] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:13:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:20.482796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:20.487321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:20.754849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:20.767923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f383e-de0a-42f9-b8e8-4d26b5004890 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:20.772028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0af0f0-ca31-4de1-a161-d279fb6faaee None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:20.796413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:20.797566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:20.797891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:20.799307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:20.959058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0d5c0141354c66bf93e05c62b55bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aad38a85bad4dbea4d518323eab5102', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:20.959877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:20.960668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:20.962564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:20.963846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:20.965745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:20.984406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:20.993752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:20.997462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:21.027916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 861/1716] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:13:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:13:21.035293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 856/1717] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:21.040689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:21.060261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d51959-06b3-4158-bdc0-cfbf5f46e238 service swift] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:21.071746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0d5c0141354c66bf93e05c62b55bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aad38a85bad4dbea4d518323eab5102', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:21.072564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:21.072946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:21.073357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:21.074415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:21.076562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:21.169166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 862/1718] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:21 2022] POST /identity/v3/projects => generated 354 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:21.174012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:21.208013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3f40d-51b5-4d4b-b419-17e18825ad00 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:21.631781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0d5c0141354c66bf93e05c62b55bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aad38a85bad4dbea4d518323eab5102', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:21.632682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:21.633608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:21.634533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:21.635210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:21.636258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:21.637975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:21.648252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:21.652017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:21.720872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 857/1719] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 548 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:21.860779 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:21.885300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:21.887992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b216de0-964d-432d-a680-0ec2d6dcc03e tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:21.896774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:21.897951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:21.898602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:21.899396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] PATH_INFO: `/v3/users/5b04e3a585e04c9abaf7037febff124b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:21.901724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b04e3a585e04c9abaf7037febff124b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:21.903934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:21.908945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d187-716f-475a-947f-b9eda73c638d tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:21.933856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d0d5c0141354c66bf93e05c62b55bcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4aad38a85bad4dbea4d518323eab5102', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:21.934441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:21.934728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:21.935263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] PATH_INFO: `/v3/projects/0dc603ec66a044e7bdf67e4c61804b25` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:21.952525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0dc603ec66a044e7bdf67e4c61804b25)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:21.962677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:22.035178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b04e3a585e04c9abaf7037febff124b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.078083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b04e3a585e04c9abaf7037febff124b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.102708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b04e3a585e04c9abaf7037febff124b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.103679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b04e3a585e04c9abaf7037febff124b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.105732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8b59955d-e438-400c-9569-5f6c313b251b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b04e3a585e04c9abaf7037febff124b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.207909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 863/1720] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:21 2022] DELETE /identity/v3/users/5b04e3a585e04c9abaf7037febff124b => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:22.211641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:22.212167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:22.213904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc603ec66a044e7bdf67e4c61804b25', 'request_id': 'req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.215977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:22.242961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc603ec66a044e7bdf67e4c61804b25', 'request_id': 'req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.292334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc603ec66a044e7bdf67e4c61804b25', 'request_id': 'req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.315119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:22.369274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:22.374720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:22.387473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] The token cache is being invalidate because project 0dc603ec66a044e7bdf67e4c61804b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:22.387836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.388609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:22.400756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 858/1721] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:21 2022] DELETE /identity/v3/projects/0dc603ec66a044e7bdf67e4c61804b25 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:22.673282 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 864/1722] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:22.980850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:22.988344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:23.507828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:23.513054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:23.537717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5339e7-70f8-412b-b11a-86d0ed9cf5e0 tempest-VolumeQuotaClassesTest-1604683465 tempest-VolumeQuotaClassesTest-1604683465-project-admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:23.566707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3132be-dbcd-4811-8c34-a1c4841f8a29 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:23.616189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:23.617206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:23.617607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:23.618024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] PATH_INFO: `/v3/users/6899853f27d342b29c44e0fea9087fae` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:23.632664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6899853f27d342b29c44e0fea9087fae)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:23.635633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:23.790638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6899853f27d342b29c44e0fea9087fae'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:23.817840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6899853f27d342b29c44e0fea9087fae'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:23.848768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6899853f27d342b29c44e0fea9087fae'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:23.849656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6899853f27d342b29c44e0fea9087fae'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:23.850621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a74b92f8-f059-4b3c-be82-06dec5feb5dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6899853f27d342b29c44e0fea9087fae'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:23.856448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:23.858776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:23.859195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:23.859485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:23.866033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:23.883901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:23.942446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:23.954627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:23.962915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 865/1723] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:23 2022] DELETE /identity/v3/users/6899853f27d342b29c44e0fea9087fae => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:23.982791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf0d04-f547-477d-b949-e744d7600bcb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:23.983623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf0d04-f547-477d-b949-e744d7600bcb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:23.983942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf0d04-f547-477d-b949-e744d7600bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:24.023640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 859/1724] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 1044 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:24.056838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2caf0d04-f547-477d-b949-e744d7600bcb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:24.058255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2caf0d04-f547-477d-b949-e744d7600bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:24.331473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 866/1725] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:24.348552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:24.350450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:24.385882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1efae-b2f7-4612-914a-d517cc475f55 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:24.560918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72324688-d55a-4b4f-84ab-0ddef0a3b9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:24.569355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324688-d55a-4b4f-84ab-0ddef0a3b9aa service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:24.570843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324688-d55a-4b4f-84ab-0ddef0a3b9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:24.575562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324688-d55a-4b4f-84ab-0ddef0a3b9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:24.654538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72324688-d55a-4b4f-84ab-0ddef0a3b9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:24.667677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72324688-d55a-4b4f-84ab-0ddef0a3b9aa service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:24.700147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 860/1726] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:25.939690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:25.941155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:25.941821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:25.994542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:26.002517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:26.026220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 867/1727] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Jul 26 17:13:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:26.086669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2e1b-ee4a-4966-a55c-d7d1bcd96fed None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:26.087227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2e1b-ee4a-4966-a55c-d7d1bcd96fed None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:26.087439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2e1b-ee4a-4966-a55c-d7d1bcd96fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:26.117731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:26.118869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:26.157047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395557fa-0b85-41f3-ab8a-63ab606e28f5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:26.170133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:26.172135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:26.172717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:26.173710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:26.189200 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-568c2e1b-ee4a-4966-a55c-d7d1bcd96fed None None] MFA Rules not processed for user `ab248df5c41d4495b9c9f46ef33c1631`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:26.191982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-568c2e1b-ee4a-4966-a55c-d7d1bcd96fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:26.201330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:26.209437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:26.237434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 868/1728] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:26.332418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:26.413394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 861/1729] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:26.419047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:26.426843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:26.662190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dd905-65a6-4d3d-b739-c397a8afc81a service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:26.678182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141db7c-9caa-4fac-81e3-6816a791d3be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:26.679006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141db7c-9caa-4fac-81e3-6816a791d3be service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:26.679467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141db7c-9caa-4fac-81e3-6816a791d3be service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:26.679842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141db7c-9caa-4fac-81e3-6816a791d3be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:26.697918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141db7c-9caa-4fac-81e3-6816a791d3be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:26.699830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141db7c-9caa-4fac-81e3-6816a791d3be service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:26.725749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 869/1730] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:26.848870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:26.849995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:26.854732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:26.973226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] MFA Rules not processed for user `97115d07acc5464bb7b097039471fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:26.976985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:27.294395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 862/1731] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:27.306860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:27.307504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:27.307594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:27.394752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] MFA Rules not processed for user `97115d07acc5464bb7b097039471fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:27.396481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:27.418640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 870/1732] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:27.440408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:27.527578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da327618-9c10-4a1a-82f0-9c2580f6deb8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:27.541703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:27.542015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:27.542349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:27.542896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:27.572183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:27.578435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:27.601671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 863/1733] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:27.903971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:27.906591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:27.991579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5605729-5f3e-4f90-8bd1-ebd0c69bc602 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:28.041272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:28.046748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:28.047052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:28.047701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] PATH_INFO: `/v3/projects/f2122cd66f434c50a00cd63b893c9b9a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:28.049975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2122cd66f434c50a00cd63b893c9b9a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:28.059733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:28.287010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:28.340003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2122cd66f434c50a00cd63b893c9b9a', 'request_id': 'req-a291ff40-d58a-48a7-8356-fbafda822d31'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.366151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:28.371168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:28.387275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2122cd66f434c50a00cd63b893c9b9a', 'request_id': 'req-a291ff40-d58a-48a7-8356-fbafda822d31'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.438862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c06760-46ce-4e63-b725-cc678adafd0c service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:28.441908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2122cd66f434c50a00cd63b893c9b9a', 'request_id': 'req-a291ff40-d58a-48a7-8356-fbafda822d31'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.519147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] The token cache is being invalidate because project f2122cd66f434c50a00cd63b893c9b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:28.524715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.530985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.562051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 871/1734] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:27 2022] DELETE /identity/v3/projects/f2122cd66f434c50a00cd63b893c9b9a => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:28.567666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:28.574889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:28.755974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291ff40-d58a-48a7-8356-fbafda822d31 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:28.769900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:28.770786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:28.771137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:28.771467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] PATH_INFO: `/v3/users/1d0d5c0141354c66bf93e05c62b55bcf` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:28.774888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0d5c0141354c66bf93e05c62b55bcf)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:28.777688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:28.838188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:28.843419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:28.845204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:28.845493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] PATH_INFO: `/v3/projects/b66d9eef7815468cb7ba69b281ddb0e1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:28.855898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66d9eef7815468cb7ba69b281ddb0e1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:28.866379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:28.910687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0d5c0141354c66bf93e05c62b55bcf'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.942247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0d5c0141354c66bf93e05c62b55bcf'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.970559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0d5c0141354c66bf93e05c62b55bcf'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.971118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0d5c0141354c66bf93e05c62b55bcf'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:28.971962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b58eaae9-a922-4dfa-b293-52f3f6d4f4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0d5c0141354c66bf93e05c62b55bcf'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.039794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 872/1735] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:28 2022] DELETE /identity/v3/users/1d0d5c0141354c66bf93e05c62b55bcf => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:29.193283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66d9eef7815468cb7ba69b281ddb0e1', 'request_id': 'req-1fd757cf-36e0-45f0-9067-f36d6bff410d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.248086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838765-b380-4822-9542-2a80fcdc52d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:29.248575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838765-b380-4822-9542-2a80fcdc52d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:29.249630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a838765-b380-4822-9542-2a80fcdc52d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:29.257524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66d9eef7815468cb7ba69b281ddb0e1', 'request_id': 'req-1fd757cf-36e0-45f0-9067-f36d6bff410d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.306576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66d9eef7815468cb7ba69b281ddb0e1', 'request_id': 'req-1fd757cf-36e0-45f0-9067-f36d6bff410d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.310341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:29.321265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7a838765-b380-4822-9542-2a80fcdc52d8 None None] MFA Rules not processed for user `c6291932ceb84b03924c0f056f29c2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:29.325820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7a838765-b380-4822-9542-2a80fcdc52d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:29.352185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] The token cache is being invalidate because project b66d9eef7815468cb7ba69b281ddb0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:29.353124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.355966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.360613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 864/1736] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:28 2022] DELETE /identity/v3/projects/b66d9eef7815468cb7ba69b281ddb0e1 => generated 0 bytes in 995 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:29.365692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:29.374446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:29.424361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:29.448597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 873/1737] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:29.622675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd757cf-36e0-45f0-9067-f36d6bff410d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:29.640254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:29.643922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:29.644936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:29.649409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] PATH_INFO: `/v3/users/597102f2fe924c07a1ef5ecba008ef5d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:29.659364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597102f2fe924c07a1ef5ecba008ef5d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:29.665353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:29.758584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f94773-2046-4a50-b276-53be48d106a0 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:29.758904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f94773-2046-4a50-b276-53be48d106a0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:29.759284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f94773-2046-4a50-b276-53be48d106a0 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:29.760895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 874/1738] 10.222.0.48 () {54 vars in 912 bytes} [Tue Jul 26 17:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:13:29.775345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:29.775663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:29.775918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:29.862138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597102f2fe924c07a1ef5ecba008ef5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.866425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:29.870149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597102f2fe924c07a1ef5ecba008ef5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.878171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:29.933418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597102f2fe924c07a1ef5ecba008ef5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:29.965971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597102f2fe924c07a1ef5ecba008ef5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:30.005998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f8cf83cc-fc96-469c-ae82-d12147aae419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597102f2fe924c07a1ef5ecba008ef5d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:30.033629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 865/1739] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:29 2022] DELETE /identity/v3/users/597102f2fe924c07a1ef5ecba008ef5d => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:30.107479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 875/1740] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:13:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:30.244095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:30.244457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:30.244648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:30.351169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] MFA Rules not processed for user `80daf046f3d34357b513374573f17791`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:30.355877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:30.375269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:30.376876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:30.407799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc7d14-09b2-4446-93b9-6e00220650d4 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:30.684528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:30.699935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14648788-7a4a-4de5-a8a0-2627d7434400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:30.700592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648788-7a4a-4de5-a8a0-2627d7434400 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:30.700879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648788-7a4a-4de5-a8a0-2627d7434400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:30.701174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14648788-7a4a-4de5-a8a0-2627d7434400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:30.729987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 866/1741] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:30.735751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648788-7a4a-4de5-a8a0-2627d7434400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:30.737323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14648788-7a4a-4de5-a8a0-2627d7434400 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:30.783015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 876/1742] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:32.012653 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:32.015150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:32.257719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbb153-84d7-4df0-8dc2-387c7afd6e4c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:32.274586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:32.275273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:32.275559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:32.282096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] PATH_INFO: `/v3/projects/4aad38a85bad4dbea4d518323eab5102` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:32.293113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aad38a85bad4dbea4d518323eab5102)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:32.301358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:32.314078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119ec0e-5309-4f3d-9a5c-59a419c95d08 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:32.314655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119ec0e-5309-4f3d-9a5c-59a419c95d08 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:32.314731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119ec0e-5309-4f3d-9a5c-59a419c95d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:32.457483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d119ec0e-5309-4f3d-9a5c-59a419c95d08 None None] MFA Rules not processed for user `459db7ff580948dc853b9f843700fe3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:32.459589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d119ec0e-5309-4f3d-9a5c-59a419c95d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:32.554634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aad38a85bad4dbea4d518323eab5102', 'request_id': 'req-09fc158b-e6d4-4624-aecf-86f0d7e458d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:32.581120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aad38a85bad4dbea4d518323eab5102', 'request_id': 'req-09fc158b-e6d4-4624-aecf-86f0d7e458d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:32.594258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aad38a85bad4dbea4d518323eab5102', 'request_id': 'req-09fc158b-e6d4-4624-aecf-86f0d7e458d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:32.633951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] The token cache is being invalidate because project 4aad38a85bad4dbea4d518323eab5102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:32.635957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:32.638024 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-09fc158b-e6d4-4624-aecf-86f0d7e458d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:32.644213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 867/1743] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:32 2022] DELETE /identity/v3/projects/4aad38a85bad4dbea4d518323eab5102 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:32.647439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d5493-7e72-43b7-b0dc-f45f348dae27 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:32.648434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d5493-7e72-43b7-b0dc-f45f348dae27 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:32.649991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d5493-7e72-43b7-b0dc-f45f348dae27 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:32.654382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 868/1744] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:13:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:13:32.656170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d5493-7e72-43b7-b0dc-f45f348dae27 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:32.666667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ef8d5493-7e72-43b7-b0dc-f45f348dae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:32.723610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 877/1745] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:32.725432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76c61e-f018-412d-ae7b-9a379788cff4 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:32.728338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76c61e-f018-412d-ae7b-9a379788cff4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:32.728822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76c61e-f018-412d-ae7b-9a379788cff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:32.799105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-df76c61e-f018-412d-ae7b-9a379788cff4 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:32.800414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-df76c61e-f018-412d-ae7b-9a379788cff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:32.920245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d5493-7e72-43b7-b0dc-f45f348dae27 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:32.926316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:32.961305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:32.961934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:32.970744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:32.971181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] PATH_INFO: `/v3/projects/7c611ec0796041b49ce93bbc7f3cef37` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:32.985814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c611ec0796041b49ce93bbc7f3cef37)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:32.987956 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:33.106699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 878/1746] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:33.109243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce0473-0017-45b0-9c8e-7ca47cd6dd63 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.109462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce0473-0017-45b0-9c8e-7ca47cd6dd63 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.109612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce0473-0017-45b0-9c8e-7ca47cd6dd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.140548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:33.174847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0cce0473-0017-45b0-9c8e-7ca47cd6dd63 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:33.178663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0cce0473-0017-45b0-9c8e-7ca47cd6dd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:33.208444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c611ec0796041b49ce93bbc7f3cef37', 'request_id': 'req-890498af-b447-44e1-85b8-f3b98ecfad09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:33.223349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c611ec0796041b49ce93bbc7f3cef37', 'request_id': 'req-890498af-b447-44e1-85b8-f3b98ecfad09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:33.238067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c611ec0796041b49ce93bbc7f3cef37', 'request_id': 'req-890498af-b447-44e1-85b8-f3b98ecfad09'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:33.283304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] The token cache is being invalidate because project 7c611ec0796041b49ce93bbc7f3cef37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:33.285006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:33.287233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-890498af-b447-44e1-85b8-f3b98ecfad09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:33.299180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 869/1747] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:32 2022] DELETE /identity/v3/projects/7c611ec0796041b49ce93bbc7f3cef37 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:33.305494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.305784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.306258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.395057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:33.396917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:33.444764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 879/1748] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:13:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:33.449384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.454978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.455320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.536559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:33.538009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:33.601949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 870/1749] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:33.609905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:33.611513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:33.649134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ae082-e30b-4f36-9748-e2e2df898dcb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:33.668564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:33.669439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.669837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.670234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.673318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:33.777160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 880/1750] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:33.781330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:33.812389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d0519-50e2-4b26-8f8d-55eb87693be1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:33.819348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:33.821436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844f173-9715-4dd8-aa48-b7157ef64ced service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:33.839792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:33.841023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.841547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.843616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.844593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:33.845110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:33.846980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 871/1751] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:33.848092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:33.857584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.857584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.857584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.892402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 881/1752] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:33.897147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:33.925907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856a698-5a4a-466f-9fa4-534680de59a2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:33.927523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:33.928908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:33.950188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 872/1753] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:33.951166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:33.952863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:33.955083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.956296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.957895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.961496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:33.973912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf14a1-b36d-4377-b1ec-d3a917b05f39 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:33.987886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:33.988616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:33.988882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:33.989191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:33.989964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:33.990279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:33.992090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:33.999118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.034604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 873/1754] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:34.038904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.054156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf0436-8d93-49fb-bbe6-61a36b61009b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:34.060473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:34.061679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.062018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.062295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.063030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:34.065734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.076364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 882/1755] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:33 2022] POST /identity/v3/projects => generated 393 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:34.088994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.140093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 874/1756] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:34 2022] POST /identity/v3/projects => generated 361 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:34.144424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.177279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:34.300286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:34.338769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85067b2-90ed-4ee2-97ef-6c40e96cac6f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:34.366295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdab085-bc72-45df-835e-64d1b2a3b543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:34.371260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdab085-bc72-45df-835e-64d1b2a3b543 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.373231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdab085-bc72-45df-835e-64d1b2a3b543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.374701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdab085-bc72-45df-835e-64d1b2a3b543 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.378251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdab085-bc72-45df-835e-64d1b2a3b543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:34.384129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdab085-bc72-45df-835e-64d1b2a3b543 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.407898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179db4-bdb7-4947-9bdd-1f9532bd9a0d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:34.424375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:34.425325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.427315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.427759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.446379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:34.448503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.477286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 875/1757] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:34 2022] POST /identity/v3/users => generated 281 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:34.483868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.484881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.485626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.491240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 883/1758] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:34 2022] POST /identity/v3/users => generated 297 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:34.492331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.547573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] MFA Rules not processed for user `97115d07acc5464bb7b097039471fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:34.552441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:34.773588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214b39f-0c9f-49b9-a5e3-4e890e01a24f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:34.794380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:34.795066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.795412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.795676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.796538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 876/1759] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:34.798013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.830461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890e5b6-587e-445a-ba51-ec97bf413482 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:34.842766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:34.843823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.844152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.844486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.845473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:34.848373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.850149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:34.853761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.870195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 877/1760] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:34.871323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.887818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa4e1f-9a7e-4c29-8e0c-20b3d38c630b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:34.898632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:34.900220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:34.901036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:34.903183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:34.904157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:34.908247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:34.931129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 878/1761] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:34.932232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.943469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 884/1762] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:34.944561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:34.985392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c812594-33d8-468f-a180-71f12076f9d0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:35.002690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:35.003352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.003638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.003915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.004639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:35.006420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:35.010822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247c25-7595-4daa-8372-9984885caea6 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:35.029120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:35.033615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.034141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.034576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.035450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:35.041342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:35.047515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 879/1763] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:35.057738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:35.082865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 885/1764] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:35.088101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:35.092077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4900ea2-3aa8-40b9-9a54-96447d255ca6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:35.111481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca069043-79bd-4c4e-8149-63fe4d343234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:35.112346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca069043-79bd-4c4e-8149-63fe4d343234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.112863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca069043-79bd-4c4e-8149-63fe4d343234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.113669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca069043-79bd-4c4e-8149-63fe4d343234 admin admin] PATH_INFO: `/v3/projects/90f2102f23cd4757ae154d36c8b3e8b6/users/349fb7c8099c4e508225bce6ecfceac0/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.127197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c1df3-94a4-4bc6-b760-c237e512aa99 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:35.136562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885d1e4-ccab-451e-8800-4de1b10ba343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:35.139651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885d1e4-ccab-451e-8800-4de1b10ba343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.139651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885d1e4-ccab-451e-8800-4de1b10ba343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.143437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885d1e4-ccab-451e-8800-4de1b10ba343 admin admin] PATH_INFO: `/v3/projects/e0d550c265414796aa7d821cf30d6f88/users/885a7761ba84432283974f223ecb199c/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.164936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca069043-79bd-4c4e-8149-63fe4d343234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f2102f23cd4757ae154d36c8b3e8b6, user_id=349fb7c8099c4e508225bce6ecfceac0, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:35.167764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca069043-79bd-4c4e-8149-63fe4d343234 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:35.220258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 880/1765] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:35 2022] PUT /identity/v3/projects/90f2102f23cd4757ae154d36c8b3e8b6/users/349fb7c8099c4e508225bce6ecfceac0/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:35.234020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.234871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.235453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.245016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885d1e4-ccab-451e-8800-4de1b10ba343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d550c265414796aa7d821cf30d6f88, user_id=885a7761ba84432283974f223ecb199c, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:35.251962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885d1e4-ccab-451e-8800-4de1b10ba343 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:35.322069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 886/1766] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:35 2022] PUT /identity/v3/projects/e0d550c265414796aa7d821cf30d6f88/users/885a7761ba84432283974f223ecb199c/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:35.333365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.333912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.334316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.427259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] MFA Rules not processed for user `349fb7c8099c4e508225bce6ecfceac0`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:35.431370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:35.480401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] MFA Rules not processed for user `885a7761ba84432283974f223ecb199c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:35.485319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:35.550863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:35.635254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 881/1767] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:35.640976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:35.698804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 887/1768] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:35.706850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:35.750961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:35.929420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a8344-e947-45f1-b531-e9f95ff48e3f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:35.944232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc4f6c-9d2e-462f-82de-db29bb125be0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:35.946036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:35.947401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.948520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.950263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.951152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:35.959583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:35.964803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:35.965639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:35.966038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:35.966420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:35.967269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:35.976252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:36.045878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 888/1769] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:35 2022] POST /identity/v3/projects => generated 391 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:36.052630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:36.069761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 882/1770] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:35 2022] POST /identity/v3/projects => generated 363 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:36.077073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:36.289715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4b2f6-683c-4162-9eaf-029eec71ad63 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:36.320083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561ca28-1b88-4f62-b9d6-8b13c6c3fa63 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:36.324606 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:36.326239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:36.327326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:36.328111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:36.334145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:36.344062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:36.345360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:36.349041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:36.352534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:36.354299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:36.359335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:36.370404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:36.464350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 889/1771] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:36 2022] POST /identity/v3/users => generated 302 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:36.469522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:36.484497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 883/1772] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:36 2022] POST /identity/v3/users => generated 288 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:36.490175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:36.512388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52121-f978-4d75-a3c4-feeb6e0d8367 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:36.527517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:36.528494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:36.528791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:36.529088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:36.532809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:36.544371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:36.565199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a9deb-72c5-4f3a-a66e-22e5e269046a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:36.576346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 890/1773] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:36.578002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:36.595918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:36.609344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:36.610239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:36.613300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:36.618375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:36.633348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:36.641853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806dc96-c28f-49ef-8a02-aae26fed3a85 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:36.689011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:36.700064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:36.702842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:36.709010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] PATH_INFO: `/v3/projects/40f67c6d2b814eb395b6b4f15189acc3/users/239f722e3e6a4addb292b44d7a5702d2/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:36.747551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 884/1774] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:13:36.750135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:36.811647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67642ada-9241-497f-a9fa-9b71c4748d0f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:36.814418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f67c6d2b814eb395b6b4f15189acc3, user_id=239f722e3e6a4addb292b44d7a5702d2, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:36.830937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:36.848681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe13f7f-3979-43e5-83e9-de532736a6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:36.854348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe13f7f-3979-43e5-83e9-de532736a6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:36.854645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe13f7f-3979-43e5-83e9-de532736a6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:36.854928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe13f7f-3979-43e5-83e9-de532736a6d9 admin admin] PATH_INFO: `/v3/projects/ba456703466f41e182ea81014996ca6e/users/ec7f92fa19434658888b41e92351461d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:36.946136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 891/1775] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:36 2022] PUT /identity/v3/projects/40f67c6d2b814eb395b6b4f15189acc3/users/239f722e3e6a4addb292b44d7a5702d2/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:36.948057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:37.010620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe13f7f-3979-43e5-83e9-de532736a6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba456703466f41e182ea81014996ca6e, user_id=ec7f92fa19434658888b41e92351461d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:37.023683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe13f7f-3979-43e5-83e9-de532736a6d9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:37.097404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 885/1776] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:36 2022] PUT /identity/v3/projects/ba456703466f41e182ea81014996ca6e/users/ec7f92fa19434658888b41e92351461d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:37.102954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07962-c786-4815-a730-ad24c01661dd None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:37.103982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07962-c786-4815-a730-ad24c01661dd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:37.104849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc07962-c786-4815-a730-ad24c01661dd None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:37.113383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 886/1777] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:13:37 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:13:37.114937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07962-c786-4815-a730-ad24c01661dd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:37.161589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:37.214035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc0992-2069-4eb8-ba9c-33e460929164 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:37.257645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:37.378723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679201bf-174e-403e-9282-8314da73a023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:37.379622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679201bf-174e-403e-9282-8314da73a023 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:37.381004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679201bf-174e-403e-9282-8314da73a023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:37.381004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679201bf-174e-403e-9282-8314da73a023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:37.381336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679201bf-174e-403e-9282-8314da73a023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:37.386335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679201bf-174e-403e-9282-8314da73a023 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:37.422913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 892/1778] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:37.425890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:37.429150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:37.429637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:37.490086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:37.495034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:37.559496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc07962-c786-4815-a730-ad24c01661dd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:37.575104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:37.577672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:37.588914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:37.589924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:37.591640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:37.596618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:37.699800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 887/1779] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 585 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:37.701660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:37.823055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ee97a-8932-4ccf-aaea-c5cc7b8cefca admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:37.863497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9452a7d-5541-4377-b0d1-8699c5f6e45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:37.864243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9452a7d-5541-4377-b0d1-8699c5f6e45d admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:37.867356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9452a7d-5541-4377-b0d1-8699c5f6e45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:37.867720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9452a7d-5541-4377-b0d1-8699c5f6e45d admin admin] PATH_INFO: `/v3/projects/40f67c6d2b814eb395b6b4f15189acc3/users/239f722e3e6a4addb292b44d7a5702d2/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:37.885316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9452a7d-5541-4377-b0d1-8699c5f6e45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f67c6d2b814eb395b6b4f15189acc3, user_id=239f722e3e6a4addb292b44d7a5702d2, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:37.885316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9452a7d-5541-4377-b0d1-8699c5f6e45d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:37.931921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 893/1780] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:13:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:37.934079 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:37.940003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 888/1781] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:37 2022] PUT /identity/v3/projects/40f67c6d2b814eb395b6b4f15189acc3/users/239f722e3e6a4addb292b44d7a5702d2/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:37.963711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1ffac-7ada-44a9-825c-b989e5038f80 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:37.971942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed61c-7757-490e-8594-7383d2723530 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:37.972246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed61c-7757-490e-8594-7383d2723530 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:37.972531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed61c-7757-490e-8594-7383d2723530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:38.074096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-105ed61c-7757-490e-8594-7383d2723530 None None] MFA Rules not processed for user `239f722e3e6a4addb292b44d7a5702d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:38.090727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-105ed61c-7757-490e-8594-7383d2723530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:38.131788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:38.133901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:38.134301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:38.136694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] PATH_INFO: `/v3/projects/ba456703466f41e182ea81014996ca6e/users/ec7f92fa19434658888b41e92351461d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:38.144102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba456703466f41e182ea81014996ca6e, user_id=ec7f92fa19434658888b41e92351461d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:38.147450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:38.189583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 894/1782] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:37 2022] PUT /identity/v3/projects/ba456703466f41e182ea81014996ca6e/users/ec7f92fa19434658888b41e92351461d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:38.201947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:38.214231 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:38.293270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:38.547063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 889/1783] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 576 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:38.548304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:38.548595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:38.548874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:38.578383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4095-9567-4fea-b167-498e77f8a1ad admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:38.602535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:38.605371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:38.605882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:38.606290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:38.634529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] MFA Rules not processed for user `ec7f92fa19434658888b41e92351461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:38.636855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:38.640575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:38.647058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:38.682344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 895/1784] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:38.686913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:38.689782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:38.830415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ddb46-add7-4e8a-9c13-f45aecc59c3b service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:38.848612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:38.849180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:38.849517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:38.851129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:38.871803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 890/1785] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:38.886748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:38.888490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:38.923160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc07135-ba21-4d3c-98ba-ffde6f41015b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:38.930814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:38.931451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:38.931746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:38.932016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:38.949484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:38.973914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:38.978668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:39.015953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 891/1786] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:39.046796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:39.048884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:39.082261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 896/1787] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:39.155545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:39.168239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81756db9-230c-4561-8df0-771c6608e31d service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:39.179105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:39.179105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:39.179105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:39.179105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:39.200994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:39.216217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be8223-3393-4cce-b072-f16afd2e3101 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:39.229271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb0a9b-7ba2-4c47-a0a0-f6b3b2bb63eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:39.230068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb0a9b-7ba2-4c47-a0a0-f6b3b2bb63eb service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:39.230354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb0a9b-7ba2-4c47-a0a0-f6b3b2bb63eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:39.230634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb0a9b-7ba2-4c47-a0a0-f6b3b2bb63eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:39.352477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:39.360325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb0a9b-7ba2-4c47-a0a0-f6b3b2bb63eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:39.364169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb0a9b-7ba2-4c47-a0a0-f6b3b2bb63eb service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:39.386494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:39.394859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:39.408118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 897/1788] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:39.426945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 892/1789] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:40.844879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:40.846102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:40.846623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:40.897542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] MFA Rules not processed for user `ab248df5c41d4495b9c9f46ef33c1631`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:40.900739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:41.028760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 898/1790] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:42.937700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:42.941756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:43.271079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e9a59-8447-45bb-8898-1df66bb60609 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:43.288733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:43.289842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:43.290390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:43.290875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] PATH_INFO: `/v3/users/051506e3708249738a7862b4dbd0eb6d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:43.293989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051506e3708249738a7862b4dbd0eb6d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:43.300906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:43.465208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051506e3708249738a7862b4dbd0eb6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:43.472244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051506e3708249738a7862b4dbd0eb6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:43.530686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051506e3708249738a7862b4dbd0eb6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:43.632797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051506e3708249738a7862b4dbd0eb6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:43.648362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051506e3708249738a7862b4dbd0eb6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:43.694339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 893/1791] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:42 2022] DELETE /identity/v3/users/051506e3708249738a7862b4dbd0eb6d => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:45.440811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:45.464571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030f6f9-c453-465d-8105-b8d0ceb81965 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:45.503184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:45.505533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:45.545402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:45.671280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f665bce-0cd4-48fa-913a-d83d5f55b165 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:45.689420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:45.690103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:45.690387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:45.690667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] PATH_INFO: `/v3/users/885a7761ba84432283974f223ecb199c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:45.691977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] RBAC: Authorizing `identity:delete_user(user_id=885a7761ba84432283974f223ecb199c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:45.692725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:45.694967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:45.695628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:45.696139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] PATH_INFO: `/v3/projects/033dca51365847a3b3cdea0018799a78` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:45.697555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:45.699041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] RBAC: Authorizing `identity:delete_project(project_id=033dca51365847a3b3cdea0018799a78)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:45.709878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:45.830642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '885a7761ba84432283974f223ecb199c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.831429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '885a7761ba84432283974f223ecb199c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.856138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '885a7761ba84432283974f223ecb199c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.876308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '885a7761ba84432283974f223ecb199c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.881372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033dca51365847a3b3cdea0018799a78', 'request_id': 'req-53d9604e-c1ce-421f-b08f-420fa731102b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.895276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-de509a79-b593-4d13-8db4-acb1929e36db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '885a7761ba84432283974f223ecb199c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.898382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033dca51365847a3b3cdea0018799a78', 'request_id': 'req-53d9604e-c1ce-421f-b08f-420fa731102b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.903331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 894/1792] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:45 2022] DELETE /identity/v3/users/885a7761ba84432283974f223ecb199c => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:45.908084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033dca51365847a3b3cdea0018799a78', 'request_id': 'req-53d9604e-c1ce-421f-b08f-420fa731102b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.913765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:45.914055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:45.914336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:45.933709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] The token cache is being invalidate because project 033dca51365847a3b3cdea0018799a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:45.938728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.947554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:45.959367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 899/1793] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:45 2022] DELETE /identity/v3/projects/033dca51365847a3b3cdea0018799a78 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:45.969463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:45.970210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:45.971467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:45.971664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:46.022036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:46.073310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9604e-c1ce-421f-b08f-420fa731102b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:46.178556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 895/1794] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:46.191779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:46.198334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:46.254887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:46.256242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:46.256534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:46.256813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] PATH_INFO: `/v3/users/80daf046f3d34357b513374573f17791` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:46.262813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=80daf046f3d34357b513374573f17791)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:46.268186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:46.292636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:46.338545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42406e4a-c1ef-452e-88e3-e8858298fb30 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:46.354447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8c951-1f6f-44d4-81ea-d8073bc61826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:46.355110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8c951-1f6f-44d4-81ea-d8073bc61826 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:46.355397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8c951-1f6f-44d4-81ea-d8073bc61826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:46.355680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8c951-1f6f-44d4-81ea-d8073bc61826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:46.366640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80daf046f3d34357b513374573f17791'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:46.376504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8c951-1f6f-44d4-81ea-d8073bc61826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:46.378015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8c951-1f6f-44d4-81ea-d8073bc61826 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:46.387666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80daf046f3d34357b513374573f17791'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:46.402658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 896/1795] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:46.402658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:46.402937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80daf046f3d34357b513374573f17791'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:46.403025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:46.403025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:46.403441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80daf046f3d34357b513374573f17791'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:46.404072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80daf046f3d34357b513374573f17791'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:46.475139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 900/1796] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:45 2022] DELETE /identity/v3/users/80daf046f3d34357b513374573f17791 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:46.548586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] MFA Rules not processed for user `459db7ff580948dc853b9f843700fe3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:46.553944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:46.709345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 897/1797] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:46.968931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:46.971403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.048305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:47.049871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.111497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb194f04-e63c-4f5d-8e85-8f9aea7de91e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:47.119443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:47.138459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:47.140044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.140535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.141366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] PATH_INFO: `/v3/projects/e0d550c265414796aa7d821cf30d6f88` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.143940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0d550c265414796aa7d821cf30d6f88)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:47.147851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:47.186872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7bb2d-93d9-4623-8423-d4d63690dbcb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:47.195015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:47.196268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.196823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.197640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] PATH_INFO: `/v3/projects/7200df1c877e407b88af53b2f32149bd` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.199406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7200df1c877e407b88af53b2f32149bd)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:47.202979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:47.284833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0d550c265414796aa7d821cf30d6f88', 'request_id': 'req-f0632358-4573-4549-8d9d-508cbe87d3f4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.300585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0d550c265414796aa7d821cf30d6f88', 'request_id': 'req-f0632358-4573-4549-8d9d-508cbe87d3f4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.331185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0d550c265414796aa7d821cf30d6f88', 'request_id': 'req-f0632358-4573-4549-8d9d-508cbe87d3f4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.357383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7200df1c877e407b88af53b2f32149bd', 'request_id': 'req-6c95aa00-e2b0-4459-a1c9-87daade94c63'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.367261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] The token cache is being invalidate because project e0d550c265414796aa7d821cf30d6f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:47.370996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.374532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f0632358-4573-4549-8d9d-508cbe87d3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.384581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 901/1798] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:46 2022] DELETE /identity/v3/projects/e0d550c265414796aa7d821cf30d6f88 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:47.384581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfe40a-de83-4ac1-abed-c9143916516c None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.384581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfe40a-de83-4ac1-abed-c9143916516c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.384581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfe40a-de83-4ac1-abed-c9143916516c None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.385532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 902/1799] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:13:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:13:47.386546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfe40a-de83-4ac1-abed-c9143916516c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:47.396782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7200df1c877e407b88af53b2f32149bd', 'request_id': 'req-6c95aa00-e2b0-4459-a1c9-87daade94c63'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.397256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5fdfe40a-de83-4ac1-abed-c9143916516c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.448509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7200df1c877e407b88af53b2f32149bd', 'request_id': 'req-6c95aa00-e2b0-4459-a1c9-87daade94c63'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.475567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] The token cache is being invalidate because project 7200df1c877e407b88af53b2f32149bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:47.475939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.479105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6c95aa00-e2b0-4459-a1c9-87daade94c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.481787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 898/1800] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:47 2022] DELETE /identity/v3/projects/7200df1c877e407b88af53b2f32149bd => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:47.491078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d107b-bcf4-4958-abdb-64529d5cf276 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.491535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d107b-bcf4-4958-abdb-64529d5cf276 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.491887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d107b-bcf4-4958-abdb-64529d5cf276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.524161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-d15d107b-bcf4-4958-abdb-64529d5cf276 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:47.525936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d15d107b-bcf4-4958-abdb-64529d5cf276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.538858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfe40a-de83-4ac1-abed-c9143916516c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:47.545632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:47.546553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.547013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.547427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] PATH_INFO: `/v3/users/239f722e3e6a4addb292b44d7a5702d2` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.548823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=239f722e3e6a4addb292b44d7a5702d2)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:47.551177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:47.579588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 899/1801] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:47.588188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d17d8-85f3-4af5-abb0-23b6b6524ee7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.588861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d17d8-85f3-4af5-abb0-23b6b6524ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.589497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d17d8-85f3-4af5-abb0-23b6b6524ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.616349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:47.621647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6e2d17d8-85f3-4af5-abb0-23b6b6524ee7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:47.624559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6e2d17d8-85f3-4af5-abb0-23b6b6524ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.648569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239f722e3e6a4addb292b44d7a5702d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.665624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239f722e3e6a4addb292b44d7a5702d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.666536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 900/1802] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:47.669498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.670143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.670670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.676846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239f722e3e6a4addb292b44d7a5702d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.677709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239f722e3e6a4addb292b44d7a5702d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.679865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239f722e3e6a4addb292b44d7a5702d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:47.705836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:47.707682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.720730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 903/1803] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:47 2022] DELETE /identity/v3/users/239f722e3e6a4addb292b44d7a5702d2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:47.721711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:47.723424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:47.731646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:47.874348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccbf99-a75e-4e7e-ae73-d8dfbdb9bddf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:47.880812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:47.881511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 901/1804] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:47.882077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.882418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.882760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.883557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:47.883864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:47.885753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:47.901019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 904/1805] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:47.906735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:47.918831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f892a8e-b4a3-4ce9-9299-da07d6623c2b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:47.925142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:47.925969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:47.926353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:47.926738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:47.928635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:47.930844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:47.978531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 902/1806] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:47 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:47.980338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:48.004011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d086d35-ff9a-4907-aee7-70bd00010680 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:48.094578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd7f44-2071-4e75-aeec-20ac8b525cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:48.095900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7f44-2071-4e75-aeec-20ac8b525cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.096547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7f44-2071-4e75-aeec-20ac8b525cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.097258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd7f44-2071-4e75-aeec-20ac8b525cb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.098696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7f44-2071-4e75-aeec-20ac8b525cb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:48.101603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd7f44-2071-4e75-aeec-20ac8b525cb8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:48.169403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:48.169771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 905/1807] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:47 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:48.193620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e51a2-c2bf-41b3-99eb-ad3ed81b9292 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:48.205406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:48.205629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.205879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.206081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.206891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:48.222095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:48.245831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 903/1808] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:13:48.250883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:48.252023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f1eed-2839-43aa-9351-eb251764d9c4 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.252444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f1eed-2839-43aa-9351-eb251764d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.252842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f1eed-2839-43aa-9351-eb251764d9c4 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.255499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 906/1809] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:13:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:13:48.269381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268679dd-914f-4c00-9c77-abef089170ba None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.269724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268679dd-914f-4c00-9c77-abef089170ba None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.269963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268679dd-914f-4c00-9c77-abef089170ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.276001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19193f94-842e-45a3-862d-7dd5e1548209 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:48.294552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:48.295415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.297728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.299308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.302603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:48.304571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:48.339176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 904/1810] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:13:48.346148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:48.375853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a9c87-841e-452a-b13e-6c79decc0471 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:48.388229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e734c82-7c69-40da-8fb5-8b765c166572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:48.389690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e734c82-7c69-40da-8fb5-8b765c166572 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.389881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-268679dd-914f-4c00-9c77-abef089170ba None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:48.390056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e734c82-7c69-40da-8fb5-8b765c166572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.390217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e734c82-7c69-40da-8fb5-8b765c166572 admin admin] PATH_INFO: `/v3/projects/7ed0b97320c94c55910a804ee0fcb2ff/users/6373c81be86e4be58cf669d2f054d2ee/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.400201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-268679dd-914f-4c00-9c77-abef089170ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:48.434492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e734c82-7c69-40da-8fb5-8b765c166572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed0b97320c94c55910a804ee0fcb2ff, user_id=6373c81be86e4be58cf669d2f054d2ee, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:48.434492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e734c82-7c69-40da-8fb5-8b765c166572 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:48.459469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 905/1811] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:48 2022] PUT /identity/v3/projects/7ed0b97320c94c55910a804ee0fcb2ff/users/6373c81be86e4be58cf669d2f054d2ee/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:48.476654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.477115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.477519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.533958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] MFA Rules not processed for user `6373c81be86e4be58cf669d2f054d2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:48.536503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:48.659206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 907/1812] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:13:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:48.724181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 906/1813] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:48.726524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268679dd-914f-4c00-9c77-abef089170ba None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:48.742604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268679dd-914f-4c00-9c77-abef089170ba None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:48.771303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:48.861437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:48.864608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:48.865696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:48.866574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:48.871166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:48.875524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:48.995037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 908/1814] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:48 2022] POST /identity/v3/projects => generated 371 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:48.999926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.019446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c526ba-a5e2-4d41-bf6f-9c1994f0af77 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.042864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:49.179637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.181473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:49.224693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.225486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.225945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.226267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.227301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.232937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.304799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 907/1815] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:48 2022] POST /identity/v3/users => generated 292 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:49.309053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.335376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b614541-3157-4f88-abf1-54894007fa5e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.356879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.357600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.358429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.358705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.359572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.361571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.387844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 908/1816] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:13:49.392555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.407783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556aa526-b353-4084-bbad-4c95c75b84dd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.429859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.429859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.430332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.430711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] PATH_INFO: `/v3/projects/483457a33a384c2c844ec6de1f141593/users/87957bb7be294726a25c39c052049449/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.469812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483457a33a384c2c844ec6de1f141593, user_id=87957bb7be294726a25c39c052049449, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.474448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.493168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80245104-fc81-480e-a40d-1df524441806 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.526070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.526997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.527474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.527799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.527958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 909/1817] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:49 2022] PUT /identity/v3/projects/483457a33a384c2c844ec6de1f141593/users/87957bb7be294726a25c39c052049449/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:49.528833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.532622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:49.546088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.548413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.566108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 909/1818] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:49.567433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.583563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898002e8-17c9-4d83-9a01-d511e56c4ae3 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.689564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9402da-3afb-49e9-a28d-4653a095f173 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.706230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.707023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.707310 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.707583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.719862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.722913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.738522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 910/1819] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:49.791315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d930672-3bcc-4fff-8c7c-4f8c4a7aa3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.792982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d930672-3bcc-4fff-8c7c-4f8c4a7aa3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.793670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d930672-3bcc-4fff-8c7c-4f8c4a7aa3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.794184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d930672-3bcc-4fff-8c7c-4f8c4a7aa3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.795889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d930672-3bcc-4fff-8c7c-4f8c4a7aa3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.800462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d930672-3bcc-4fff-8c7c-4f8c4a7aa3e5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.831969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 910/1820] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:49.837957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:49.850285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5d15-ac40-4b2b-b17b-d2e316d87705 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:49.860652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:49.867031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.867700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.869883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] PATH_INFO: `/v3/projects/483457a33a384c2c844ec6de1f141593/users/87957bb7be294726a25c39c052049449/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:49.879919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=483457a33a384c2c844ec6de1f141593, user_id=87957bb7be294726a25c39c052049449, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:49.894356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:49.930968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 911/1821] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:49 2022] PUT /identity/v3/projects/483457a33a384c2c844ec6de1f141593/users/87957bb7be294726a25c39c052049449/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:49.953986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:49.954619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:49.955378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:50.002031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] MFA Rules not processed for user `87957bb7be294726a25c39c052049449`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:50.006503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:50.033661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:50.137465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 911/1822] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:50.149918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:50.151810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:50.231398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:50.256167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520cf62-f2c3-4afc-8ab4-0daad73bcc9b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:50.261401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c9205-374a-4732-8227-591747993c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:50.262148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c9205-374a-4732-8227-591747993c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:50.262610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c9205-374a-4732-8227-591747993c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:50.263271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c9205-374a-4732-8227-591747993c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:50.299558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:50.302025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:50.319445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c9205-374a-4732-8227-591747993c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:50.323763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c9205-374a-4732-8227-591747993c04 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:50.357577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 912/1823] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:50.534145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc659b3-b4f4-4243-bbc8-7572036bc900 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:50.541905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:50.543000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:50.543985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:50.544521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] PATH_INFO: `/v3/projects/40f67c6d2b814eb395b6b4f15189acc3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:50.546424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f67c6d2b814eb395b6b4f15189acc3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:50.551821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:50.567329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c9205-374a-4732-8227-591747993c04 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:50.583480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c9205-374a-4732-8227-591747993c04 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:50.593719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da80721-f22c-4aa9-a8f7-163d1a43ac32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:50.594949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da80721-f22c-4aa9-a8f7-163d1a43ac32 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:50.595589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da80721-f22c-4aa9-a8f7-163d1a43ac32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:50.596102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da80721-f22c-4aa9-a8f7-163d1a43ac32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:50.712786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f67c6d2b814eb395b6b4f15189acc3', 'request_id': 'req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:50.744669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f67c6d2b814eb395b6b4f15189acc3', 'request_id': 'req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:50.777247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f67c6d2b814eb395b6b4f15189acc3', 'request_id': 'req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:50.826167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da80721-f22c-4aa9-a8f7-163d1a43ac32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:50.828306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da80721-f22c-4aa9-a8f7-163d1a43ac32 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:50.872549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 913/1824] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:13:50.881824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] The token cache is being invalidate because project 40f67c6d2b814eb395b6b4f15189acc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:50.882341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:50.898478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a2130952-8e9b-4c3e-bfec-2ad7bbb27f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:50.912645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 912/1825] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:50 2022] DELETE /identity/v3/projects/40f67c6d2b814eb395b6b4f15189acc3 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:50.916578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:50.916873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:50.917185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:50.966461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:50.968058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:51.161228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 914/1826] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:51.169094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.169391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.169678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.211267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:51.213699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:51.226652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 913/1827] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:51.230706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:51.253722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5bff01-05f0-462d-8fc6-561436d88286 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:51.267098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:51.268664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.281006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.281464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.283376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:51.284013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:51.288592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:51.321862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 915/1828] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:51.323303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:51.341118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23a51-57ac-4779-baa2-363c1a395b73 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:51.355164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:51.356487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.356727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.357078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.359731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:51.362410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:51.421287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 914/1829] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:51 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:51.432622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:51.476884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56ecaf-bb0d-4cf8-b831-e23c65538574 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:51.649065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:51.651039 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.651687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.653009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.654654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:51.661832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:51.709524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:51.721500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 916/1830] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:51 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:51.724151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:51.767857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb4ca2-ba13-4933-bf29-010be759ffe5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:51.793644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:51.794347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.794769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.795055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.796222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:51.798082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:51.844188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 915/1831] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:51.848170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:51.867921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dbfc35-37c8-4371-9f14-85f52050aedc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:51.881785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d2e0a-475b-4c96-8af5-4ca1b51982d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:51.882526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2e0a-475b-4c96-8af5-4ca1b51982d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.883611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2e0a-475b-4c96-8af5-4ca1b51982d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.883611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d2e0a-475b-4c96-8af5-4ca1b51982d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.883847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2e0a-475b-4c96-8af5-4ca1b51982d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:51.887884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d2e0a-475b-4c96-8af5-4ca1b51982d8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:51.901826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 917/1832] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:51.907783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:51.915907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:51.926973 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3760385-82bc-4fcb-84ef-37835d8350ca admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:51.938840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:51.939669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:51.940003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:51.940627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] PATH_INFO: `/v3/projects/7dee8510af154587b12078110c5da3e0/users/cc5f13de7034440cbeeb4a6be9d865d7/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:51.977044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dee8510af154587b12078110c5da3e0, user_id=cc5f13de7034440cbeeb4a6be9d865d7, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:51.979830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:52.009325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 916/1833] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:51 2022] PUT /identity/v3/projects/7dee8510af154587b12078110c5da3e0/users/cc5f13de7034440cbeeb4a6be9d865d7/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:52.021375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.021733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.022101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:52.074761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] MFA Rules not processed for user `cc5f13de7034440cbeeb4a6be9d865d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:52.076146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:52.259877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:52.262012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:52.268753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 918/1834] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:52.270856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:52.389812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0abd2f-126b-4b6d-b503-de6eba8f69d8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:52.413406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:52.414400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.414829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.415666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:52.416873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:52.421125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:52.470301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 919/1835] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:52 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:52.476166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:52.486145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c198024-cb9a-480a-b087-1d596396c78a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:52.651416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:52.655210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.655841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.656309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] PATH_INFO: `/v3/users/349fb7c8099c4e508225bce6ecfceac0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:52.657269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d749-1b4d-4e5d-9365-d276fae920f0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:52.662311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349fb7c8099c4e508225bce6ecfceac0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:52.668410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:52.674615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:52.675614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.676112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.676590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:52.677536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:52.683791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:52.710861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:52.737219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 920/1836] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:52 2022] POST /identity/v3/users => generated 301 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:52.740532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:52.793516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349fb7c8099c4e508225bce6ecfceac0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:52.804508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab84ecc-2ed4-48a5-a746-0c0020845d98 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:52.814854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349fb7c8099c4e508225bce6ecfceac0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:52.828162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:52.829167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.829635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.830156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:52.831444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:52.833492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:52.836733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349fb7c8099c4e508225bce6ecfceac0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:52.838618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349fb7c8099c4e508225bce6ecfceac0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:52.840466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-54771103-0ba7-4630-b9b9-cf41e7c40d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349fb7c8099c4e508225bce6ecfceac0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:52.862041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 921/1837] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:52.866453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:52.870979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:52.889357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 917/1838] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:52 2022] DELETE /identity/v3/users/349fb7c8099c4e508225bce6ecfceac0 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:52.890853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598479ae-f29d-484a-bae9-a8b7ec9ef2f3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:52.897225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.897539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.897816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:52.915964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:52.958378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:52.960994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:52.990421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:52.991132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:52.991416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:52.991804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] PATH_INFO: `/v3/projects/11794772814e4056a47ae758cbded741/users/03c88852fff34dd1b5868bbe1dd3b964/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:53.017010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11794772814e4056a47ae758cbded741, user_id=03c88852fff34dd1b5868bbe1dd3b964, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:53.020481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:53.040906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 922/1839] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:52 2022] PUT /identity/v3/projects/11794772814e4056a47ae758cbded741/users/03c88852fff34dd1b5868bbe1dd3b964/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:53.045004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:53.098700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 918/1840] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:53.138275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:53.143109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:53.153527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f11534-8004-4fe0-8d82-81bcee48bea3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:53.159967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:53.160674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:53.161181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:53.161453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:53.162179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:53.163972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:53.182720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 923/1841] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:53.187087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:53.219520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe3375-fdc2-464a-917c-f1ec37fa350e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:53.237002 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bea9bb-fe8a-4ea6-8dbc-7aa77148ac37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:53.238515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bea9bb-fe8a-4ea6-8dbc-7aa77148ac37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:53.239217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bea9bb-fe8a-4ea6-8dbc-7aa77148ac37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:53.242392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bea9bb-fe8a-4ea6-8dbc-7aa77148ac37 admin admin] PATH_INFO: `/v3/projects/11794772814e4056a47ae758cbded741/users/03c88852fff34dd1b5868bbe1dd3b964/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:53.245971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bea9bb-fe8a-4ea6-8dbc-7aa77148ac37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11794772814e4056a47ae758cbded741, user_id=03c88852fff34dd1b5868bbe1dd3b964, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:53.250113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bea9bb-fe8a-4ea6-8dbc-7aa77148ac37 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:53.280697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 924/1842] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:53 2022] PUT /identity/v3/projects/11794772814e4056a47ae758cbded741/users/03c88852fff34dd1b5868bbe1dd3b964/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:53.296475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:53.296831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:53.297122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:53.375281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] MFA Rules not processed for user `03c88852fff34dd1b5868bbe1dd3b964`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:53.377970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:53.446542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:53.452076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3a149-bcc8-41bc-93d5-9d38235eef4d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:53.526055 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 925/1843] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:53.549165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:53.550868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:53.671569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f27e35-0f08-409b-ae77-1ce1573b8b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:53.673331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f27e35-0f08-409b-ae77-1ce1573b8b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:53.673818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f27e35-0f08-409b-ae77-1ce1573b8b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:53.674307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f27e35-0f08-409b-ae77-1ce1573b8b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:53.740897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a7390-6a53-4ee3-a094-f56c86219c6f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:53.752936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:53.756595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:53.756879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:53.760943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:53.809527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:53.818987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:53.842839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 926/1844] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:53.987529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f27e35-0f08-409b-ae77-1ce1573b8b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:53.990269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f27e35-0f08-409b-ae77-1ce1573b8b67 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:54.023229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 919/1845] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 886 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:54.199737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:54.220865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce2f5d-c3af-4c05-963e-be5f3f179bb7 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:54.228992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:54.230223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:54.230846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:54.231397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] PATH_INFO: `/v3/users/cc5f13de7034440cbeeb4a6be9d865d7` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:54.233867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc5f13de7034440cbeeb4a6be9d865d7)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:54.237815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:54.344234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc5f13de7034440cbeeb4a6be9d865d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:54.345377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc5f13de7034440cbeeb4a6be9d865d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:54.367470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc5f13de7034440cbeeb4a6be9d865d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:54.379596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc5f13de7034440cbeeb4a6be9d865d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:54.391000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc5f13de7034440cbeeb4a6be9d865d7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:54.403748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 927/1846] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:54 2022] DELETE /identity/v3/users/cc5f13de7034440cbeeb4a6be9d865d7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:54.410708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:54.410840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:54.411219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:54.444394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:54.446531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:54.453975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:54.586705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 920/1847] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:54.594748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:54.597206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:54.703713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:54.720737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d803e8d-0866-4531-a1cc-f91266a47c9f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:54.734030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:54.734352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:54.734597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:54.734879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:54.761600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:54.766658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:54.801311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 928/1848] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:13:54.844180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:54.846551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:54.869955 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082451c-831a-455b-9a89-97f20a3de2d8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:54.886701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:54.887501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:54.888492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:54.889126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] PATH_INFO: `/v3/projects/90f2102f23cd4757ae154d36c8b3e8b6` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:54.900785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f2102f23cd4757ae154d36c8b3e8b6)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:54.903016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:55.056715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f2102f23cd4757ae154d36c8b3e8b6', 'request_id': 'req-7c655472-fef9-4534-9d2b-513c61faa601'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.082956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f2102f23cd4757ae154d36c8b3e8b6', 'request_id': 'req-7c655472-fef9-4534-9d2b-513c61faa601'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.092633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f2102f23cd4757ae154d36c8b3e8b6', 'request_id': 'req-7c655472-fef9-4534-9d2b-513c61faa601'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.131324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] The token cache is being invalidate because project 90f2102f23cd4757ae154d36c8b3e8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:55.134625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.137434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.148936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 921/1849] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:54 2022] DELETE /identity/v3/projects/90f2102f23cd4757ae154d36c8b3e8b6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:55.150541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:55.166989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf3d33c-39b7-4a16-93c9-7bd1147fd666 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:55.387765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:55.391240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:55.391624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:55.392130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] PATH_INFO: `/v3/users/ec7f92fa19434658888b41e92351461d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:55.394733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7f92fa19434658888b41e92351461d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:55.398135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:55.516811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7f92fa19434658888b41e92351461d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.518777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7f92fa19434658888b41e92351461d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.553068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7f92fa19434658888b41e92351461d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.570333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7f92fa19434658888b41e92351461d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.619636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7f92fa19434658888b41e92351461d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:55.642139 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 929/1850] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:55 2022] DELETE /identity/v3/users/ec7f92fa19434658888b41e92351461d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:55.810096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:55.811793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:55.851100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c655472-fef9-4534-9d2b-513c61faa601 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:56.034138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:56.035204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:56.035670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:56.036136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] PATH_INFO: `/v3/projects/7dee8510af154587b12078110c5da3e0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:56.038272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dee8510af154587b12078110c5da3e0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:56.041863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:56.149341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:56.181230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dee8510af154587b12078110c5da3e0', 'request_id': 'req-5c95bbee-3732-450d-8c4a-58dc47aab1f7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.205992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dee8510af154587b12078110c5da3e0', 'request_id': 'req-5c95bbee-3732-450d-8c4a-58dc47aab1f7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.231034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dee8510af154587b12078110c5da3e0', 'request_id': 'req-5c95bbee-3732-450d-8c4a-58dc47aab1f7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.273033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] The token cache is being invalidate because project 7dee8510af154587b12078110c5da3e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:56.273722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.274770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.282355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:56.283130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 922/1851] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:55 2022] DELETE /identity/v3/projects/7dee8510af154587b12078110c5da3e0 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:56.284199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:56.312445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3069f-0053-4474-b94a-d8efd1567e2c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:56.312783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:56.315579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:56.477717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:56.480293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95bbee-3732-450d-8c4a-58dc47aab1f7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:56.480918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:56.485496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:56.486362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:56.487191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:56.487313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:56.487589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] PATH_INFO: `/v3/projects/ba456703466f41e182ea81014996ca6e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:56.487944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] PATH_INFO: `/v3/users/03c88852fff34dd1b5868bbe1dd3b964` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:56.489361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba456703466f41e182ea81014996ca6e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:56.491665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:56.491832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c88852fff34dd1b5868bbe1dd3b964)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:56.497392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:56.528418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:56.586410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba456703466f41e182ea81014996ca6e', 'request_id': 'req-c472c1ad-af3a-44db-a051-48d1e85946cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.597866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba456703466f41e182ea81014996ca6e', 'request_id': 'req-c472c1ad-af3a-44db-a051-48d1e85946cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.607914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba456703466f41e182ea81014996ca6e', 'request_id': 'req-c472c1ad-af3a-44db-a051-48d1e85946cb'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.649936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] The token cache is being invalidate because project ba456703466f41e182ea81014996ca6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:56.651672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.656360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c472c1ad-af3a-44db-a051-48d1e85946cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.663991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 923/1852] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:56 2022] DELETE /identity/v3/projects/ba456703466f41e182ea81014996ca6e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:56.669484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:56.669837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:56.670195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:56.677575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c88852fff34dd1b5868bbe1dd3b964'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.678390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c88852fff34dd1b5868bbe1dd3b964'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.715729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:56.718278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:56.720250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c88852fff34dd1b5868bbe1dd3b964'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.744353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c88852fff34dd1b5868bbe1dd3b964'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.761245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8fe7e6f9-3e75-46f8-b90f-919a432336ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c88852fff34dd1b5868bbe1dd3b964'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:56.778472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 930/1853] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:13:56 2022] DELETE /identity/v3/users/03c88852fff34dd1b5868bbe1dd3b964 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:56.837820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:56.876588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 924/1854] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:56.892410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:56.892919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:56.893805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:56.933890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:56.936234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:57.095079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 931/1855] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:57.096802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.114359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34f92b-bdd1-4640-8dc9-a90d329cac83 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.119859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.120672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.121035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.121466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.122182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:57.122565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.124492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.145624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 925/1856] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:57.154058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.169652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00d86a-06f0-47c8-b903-1ec82ad4ea88 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.194631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.195290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.195599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.195969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.197104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.198900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.272335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 932/1857] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:57 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:57.274570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.303594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a548835-0cd4-413d-a358-03acce42f7f8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.455292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.456056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.456347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.456817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.458091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.460125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.497771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 926/1858] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:57 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:57.499872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.511314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206ed9f-496b-4843-9991-f5504bb23342 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.523228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.523878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.524162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.524438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.525179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.526883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.547822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 933/1859] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:57.552699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.574693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0831f7-a3b7-427a-8f35-7821b38bd727 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.582092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.584053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:57.586584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.587830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.588447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.590166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.593056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.596094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.600515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137788f-d9f3-4784-971e-2c5653d1fbae admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.605779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.606580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.606925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.607267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] PATH_INFO: `/v3/projects/11794772814e4056a47ae758cbded741` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.608744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11794772814e4056a47ae758cbded741)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.610792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.621679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 927/1860] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:57.628208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.649966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e986c5d-db58-4d55-8bec-3290bd991e62 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.659971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45699257-1d6f-441d-886c-692fc7037ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.661148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45699257-1d6f-441d-886c-692fc7037ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.661666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45699257-1d6f-441d-886c-692fc7037ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.662182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45699257-1d6f-441d-886c-692fc7037ee8 admin admin] PATH_INFO: `/v3/projects/b6535b43248f4199b647722d719a1f8c/users/5475ae4b1f2043fb9530013b4a828ebe/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.685366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11794772814e4056a47ae758cbded741', 'request_id': 'req-121fcc9b-a1b7-413e-89be-4b179cd5efa4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:57.687842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45699257-1d6f-441d-886c-692fc7037ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6535b43248f4199b647722d719a1f8c, user_id=5475ae4b1f2043fb9530013b4a828ebe, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.688767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:57.690890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45699257-1d6f-441d-886c-692fc7037ee8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:57.699927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11794772814e4056a47ae758cbded741', 'request_id': 'req-121fcc9b-a1b7-413e-89be-4b179cd5efa4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:57.708943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 928/1861] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:57 2022] PUT /identity/v3/projects/b6535b43248f4199b647722d719a1f8c/users/5475ae4b1f2043fb9530013b4a828ebe/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:57.715712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11794772814e4056a47ae758cbded741', 'request_id': 'req-121fcc9b-a1b7-413e-89be-4b179cd5efa4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:57.725837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.726147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.726467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.741375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] The token cache is being invalidate because project 11794772814e4056a47ae758cbded741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:13:57.741589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:57.742727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-121fcc9b-a1b7-413e-89be-4b179cd5efa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:13:57.744716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 934/1862] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:13:57 2022] DELETE /identity/v3/projects/11794772814e4056a47ae758cbded741 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:57.751976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766d07-70a9-421f-a70d-6bdc8258e0d4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.752445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766d07-70a9-421f-a70d-6bdc8258e0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.752890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766d07-70a9-421f-a70d-6bdc8258e0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.780495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] MFA Rules not processed for user `5475ae4b1f2043fb9530013b4a828ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:57.781661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-cf766d07-70a9-421f-a70d-6bdc8258e0d4 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:57.782745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:57.783474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cf766d07-70a9-421f-a70d-6bdc8258e0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:57.897365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 929/1863] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:57.905262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.905811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:57.911176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 935/1864] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:57.922687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.923265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.923719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.950372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:57.952087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:57.965119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 936/1865] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:57.966202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:57.973109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544ca0c-20c6-43d0-a670-697479520d6e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.980691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13376dd4-7d6f-457e-b080-a9c53307a793 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:57.988118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:57.989254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:57.989723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:57.990213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:57.991137 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:13:57.991632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:57.993528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.025319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 937/1866] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:13:58.031428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.032323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.041766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.042191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.042474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.043459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.047326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.057525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db613a4-2f69-453f-8478-06b06ce8b894 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.069357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.071091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.072190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.073152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.074912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.079626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.107742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 930/1867] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:57 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:58.114323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.137403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:58.163569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 938/1868] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:58 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:13:58.165934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.417242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b467924-3e56-404f-8f81-cd38351cf50f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.468397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaaabc-bfcc-4ca6-95f1-4711938fdc2d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.491415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.492434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.492906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.493425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.495019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.498423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.555679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.558961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.560128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.561108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.564665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.574175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 939/1869] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:58 2022] POST /identity/v3/users => generated 290 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:58.578388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.598681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.635079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0eac3-65ea-489b-8e8a-df03ed9b2882 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.647418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 931/1870] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:58 2022] POST /identity/v3/users => generated 297 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:58.648399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.660743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6dce8-6398-450a-8fbd-5dc750d7298c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.662865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.663851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.664311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.665067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.666227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.666368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.666871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.667153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.667486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.668138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.670574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.681487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.699380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 932/1871] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:58.700430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.711601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6a67c-b9c0-4087-a504-1acf737b9fbc admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.715104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 940/1872] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:58.720276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.731964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.734223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43ac87-8a78-46f4-b279-c8ba41d2e47d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.734609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.734990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.735277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] PATH_INFO: `/v3/projects/3648242a45af45adbea4176139c6edec/users/fcd0646587d74d73acb886cb62ed088b/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.746533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.747227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.747509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.747786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.748545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.751956 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.769365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3648242a45af45adbea4176139c6edec, user_id=fcd0646587d74d73acb886cb62ed088b, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.775985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:58.785725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 941/1873] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:13:58.792183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.812746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:58.817813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 933/1874] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:58 2022] PUT /identity/v3/projects/3648242a45af45adbea4176139c6edec/users/fcd0646587d74d73acb886cb62ed088b/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:13:58.820048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:58.827203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c263da-7a9c-48dc-be51-439572b3b586 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:58.946764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bead606-fd0a-4700-b40b-5bc5faebb87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:58.948827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead606-fd0a-4700-b40b-5bc5faebb87b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:58.949464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead606-fd0a-4700-b40b-5bc5faebb87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:58.949938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bead606-fd0a-4700-b40b-5bc5faebb87b admin admin] PATH_INFO: `/v3/projects/c15c132af6024326a0ef763397b66d44/users/fd6ab8a404414dcb89eff2145e46c19c/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:58.974837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bead606-fd0a-4700-b40b-5bc5faebb87b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15c132af6024326a0ef763397b66d44, user_id=fd6ab8a404414dcb89eff2145e46c19c, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:58.978349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bead606-fd0a-4700-b40b-5bc5faebb87b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:59.001482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d360-1f31-4214-b19e-7fad15b3298e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:59.002982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 942/1875] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:58 2022] PUT /identity/v3/projects/c15c132af6024326a0ef763397b66d44/users/fd6ab8a404414dcb89eff2145e46c19c/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:59.022694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.023435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.023992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.045693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:59.098863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] MFA Rules not processed for user `fd6ab8a404414dcb89eff2145e46c19c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:59.100996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:59.193551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:59.194559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.195356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.195829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.197062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:59.199642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:59.222219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 934/1876] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:13:59.227035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:59.244897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3495b-7d2f-4fed-bbd2-f63a428e1036 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:59.251560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc09e67-5c30-4a6b-b3d5-03090954f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:59.254462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09e67-5c30-4a6b-b3d5-03090954f86c admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.255147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09e67-5c30-4a6b-b3d5-03090954f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.255770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc09e67-5c30-4a6b-b3d5-03090954f86c admin admin] PATH_INFO: `/v3/projects/3648242a45af45adbea4176139c6edec/users/fcd0646587d74d73acb886cb62ed088b/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.261198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc09e67-5c30-4a6b-b3d5-03090954f86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3648242a45af45adbea4176139c6edec, user_id=fcd0646587d74d73acb886cb62ed088b, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:59.266804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc09e67-5c30-4a6b-b3d5-03090954f86c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:59.291952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 935/1877] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:13:59 2022] PUT /identity/v3/projects/3648242a45af45adbea4176139c6edec/users/fcd0646587d74d73acb886cb62ed088b/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:13:59.306390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.307457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.308400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.327872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 943/1878] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:13:59.340686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:59.381000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114d3c1-82cd-4892-92e7-62936afd75e4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:59.398762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] MFA Rules not processed for user `fcd0646587d74d73acb886cb62ed088b`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:13:59.400483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:59.523964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:59.524620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.524902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.525202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.525880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:59.528069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:59.603702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 936/1879] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:13:59.649405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 944/1880] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:13:59 2022] POST /identity/v3/projects => generated 379 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:59.655514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:59.678340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6e777-781c-4e7c-8bba-f0e6daa215cb None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:59.679138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:59.681598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:13:59.691284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:13:59.858560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1f0b-c638-4554-9513-208f25ce9261 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:13:59.869963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:59.870809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.871145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.871466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.872223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:13:59.875435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:13:59.895479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:13:59.895743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:13:59.897977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:13:59.898671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:13:59.930152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 937/1881] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:13:59 2022] POST /identity/v3/users => generated 290 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:13:59.935743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:13:59.970326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e8e4a-630a-48bb-9ea5-295069b8d361 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:00.003212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:00.003717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:00.004017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:00.004291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:00.005094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:00.006817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:00.037910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:00.053547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 938/1882] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:00.060572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:00.105831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:00.111676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:00.113570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df2f50-0ca2-4016-bc17-f297ef288c47 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:00.138586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a399c7-11cb-4203-8d67-0b7068b631bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:00.141430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a399c7-11cb-4203-8d67-0b7068b631bb admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:00.142957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a399c7-11cb-4203-8d67-0b7068b631bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:00.144645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a399c7-11cb-4203-8d67-0b7068b631bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:00.148001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a399c7-11cb-4203-8d67-0b7068b631bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:00.152340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a399c7-11cb-4203-8d67-0b7068b631bb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:00.153068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 945/1883] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:13:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:00.189413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 939/1884] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:00.194150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:00.215316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b135a-e04d-4e0e-be8f-87718b1db0b0 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:00.228776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:00.229453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:00.229737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:00.230013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] PATH_INFO: `/v3/projects/f2920cb8a4c74818bd8be9394bf4fd6f/users/5dca86732e7447e38501b266958cc8a1/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:00.257832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2920cb8a4c74818bd8be9394bf4fd6f, user_id=5dca86732e7447e38501b266958cc8a1, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:00.260739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:00.307594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 946/1885] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:00 2022] PUT /identity/v3/projects/f2920cb8a4c74818bd8be9394bf4fd6f/users/5dca86732e7447e38501b266958cc8a1/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:00.328435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:00.328982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:00.329283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:00.426578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] MFA Rules not processed for user `5dca86732e7447e38501b266958cc8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:00.430396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:00.585697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:00.588199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:00.663515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 940/1886] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:00.664652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:00.767791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333c2f5-502f-499c-a0a2-b6dd5ca9553c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:00.793645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:00.794475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:00.794805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:00.795861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:00.936355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948a5e-18d9-460a-b7d5-ea8eb7067f7d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:00.959771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:00.960694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:00.961240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:00.962394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:00.962830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:00.969883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:00.997342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:01.042098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:01.047920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:01.051062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 941/1887] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:00 2022] POST /identity/v3/projects => generated 379 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:01.057515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:01.081393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 947/1888] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:01.163968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273dce8-5ad2-4311-b264-c2c79299deb2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:01.170001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:01.171050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:01.171345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:01.171624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:01.172331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:01.174590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:01.188580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:01.214031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 942/1889] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:01 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:01.219413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:01.238188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0aaf1-0b2d-44e3-bddb-121a5b549b1c service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:01.255576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:01.256238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:01.256588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:01.256840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:01.257650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:01.259394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:01.284106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 948/1890] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:01.291395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:01.310309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d182409-2428-4541-9741-de6b916216ac admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:01.331136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:01.331897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:01.334153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:01.335743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] PATH_INFO: `/v3/projects/0109633806ec473fb2eb7feb9ce7859b/users/e2472be472a14070814e7294fb9b717d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:01.383989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0109633806ec473fb2eb7feb9ce7859b, user_id=e2472be472a14070814e7294fb9b717d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:01.406266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:01.446472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 943/1891] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:01 2022] PUT /identity/v3/projects/0109633806ec473fb2eb7feb9ce7859b/users/e2472be472a14070814e7294fb9b717d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:01.452866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:01.484275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305626d-65f7-4a0f-a65d-41ee807578a2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:01.730041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d1cfa-c88f-40e8-8f60-eb6963ebabe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:01.730702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d1cfa-c88f-40e8-8f60-eb6963ebabe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:01.730997 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d1cfa-c88f-40e8-8f60-eb6963ebabe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:01.731281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d1cfa-c88f-40e8-8f60-eb6963ebabe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:01.736890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d1cfa-c88f-40e8-8f60-eb6963ebabe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:01.738333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d1cfa-c88f-40e8-8f60-eb6963ebabe1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:01.781199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:01.783741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 949/1892] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:01.817892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd7630-f33a-4f90-b3b7-3574feb78b26 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:01.832422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:01.835985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:01.836432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:01.836696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] PATH_INFO: `/v3/projects/0109633806ec473fb2eb7feb9ce7859b/users/e2472be472a14070814e7294fb9b717d/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:01.843212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0109633806ec473fb2eb7feb9ce7859b, user_id=e2472be472a14070814e7294fb9b717d, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:01.850524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:01.886377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 944/1893] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:01 2022] PUT /identity/v3/projects/0109633806ec473fb2eb7feb9ce7859b/users/e2472be472a14070814e7294fb9b717d/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:01.904339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3777-76fb-44a8-95ac-235cf0a29d34 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:01.905595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3777-76fb-44a8-95ac-235cf0a29d34 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:01.906190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3777-76fb-44a8-95ac-235cf0a29d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:01.923736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:01.996713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4e2e3777-76fb-44a8-95ac-235cf0a29d34 None None] MFA Rules not processed for user `e2472be472a14070814e7294fb9b717d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:01.998810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4e2e3777-76fb-44a8-95ac-235cf0a29d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:02.069308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244711a-d565-4189-918e-fe981fd59a35 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:02.081971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:02.082621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:02.082903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:02.083179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] PATH_INFO: `/v3/users/5475ae4b1f2043fb9530013b4a828ebe` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:02.086968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5475ae4b1f2043fb9530013b4a828ebe)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:02.088977 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:02.224659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5475ae4b1f2043fb9530013b4a828ebe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:02.247980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5475ae4b1f2043fb9530013b4a828ebe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:02.267646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5475ae4b1f2043fb9530013b4a828ebe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:02.268874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5475ae4b1f2043fb9530013b4a828ebe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:02.269699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5475ae4b1f2043fb9530013b4a828ebe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:02.287550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 950/1894] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:02.313467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 945/1895] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:01 2022] DELETE /identity/v3/users/5475ae4b1f2043fb9530013b4a828ebe => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:02.319802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:02.320868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:02.321010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:02.387807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:02.391286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:02.417021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:02.419652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:02.551412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:02.599235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942cff0-cc50-4ad6-b408-17ebd4c6c748 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:02.611097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:02.611781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:02.612065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:02.612506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:02.615041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:02.658516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:02.682690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 951/1896] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:02.731598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:02.735387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:02.755467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 946/1897] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:02.760413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:02.762531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:02.903303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3196490-ed0f-4719-8489-c2e921fcb543 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:02.909385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:02.910910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:02.911206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:02.911534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:02.934456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:02.938295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:02.955424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 952/1898] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:03.041697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:03.055110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60911-2ec8-4d03-a609-18e23efeacdc service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:03.065853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:03.066546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:03.066891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:03.067244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:03.069521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:03.207526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:03.209932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:03.237913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 947/1899] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:04.183950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:04.185570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:04.203231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd102c1b-8dc0-47cf-b773-ba1bcce3363c service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:04.210277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:04.210950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:04.211423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:04.211705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] PATH_INFO: `/v3/projects/b6535b43248f4199b647722d719a1f8c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:04.213091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6535b43248f4199b647722d719a1f8c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:04.215102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:04.310595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6535b43248f4199b647722d719a1f8c', 'request_id': 'req-b6220a81-5d7b-4af0-a354-94898a90c79b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.323905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6535b43248f4199b647722d719a1f8c', 'request_id': 'req-b6220a81-5d7b-4af0-a354-94898a90c79b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.336416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6535b43248f4199b647722d719a1f8c', 'request_id': 'req-b6220a81-5d7b-4af0-a354-94898a90c79b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.363726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] The token cache is being invalidate because project b6535b43248f4199b647722d719a1f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:04.364432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.365845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.368694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 953/1900] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:04 2022] DELETE /identity/v3/projects/b6535b43248f4199b647722d719a1f8c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:04.374206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:04.395092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c936a8-a10c-47e8-be91-899e1084b3f3 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:04.464141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:04.464802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:04.465120 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:04.465387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] PATH_INFO: `/v3/users/fcd0646587d74d73acb886cb62ed088b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:04.466683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd0646587d74d73acb886cb62ed088b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:04.469381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:04.502829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:04.537126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd0646587d74d73acb886cb62ed088b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.546940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd0646587d74d73acb886cb62ed088b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.557544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd0646587d74d73acb886cb62ed088b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.558289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd0646587d74d73acb886cb62ed088b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.559850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-91d2dfbf-36fc-467a-bc11-b2f5b01b3023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd0646587d74d73acb886cb62ed088b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:04.583103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 948/1901] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:04 2022] DELETE /identity/v3/users/fcd0646587d74d73acb886cb62ed088b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:05.126381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:05.130175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:05.155042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6220a81-5d7b-4af0-a354-94898a90c79b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:05.279626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:05.280820 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.281367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.281853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] PATH_INFO: `/v3/projects/3648242a45af45adbea4176139c6edec` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.303967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3648242a45af45adbea4176139c6edec)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:05.309776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:05.371643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:05.411450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3648242a45af45adbea4176139c6edec', 'request_id': 'req-65771807-28b3-432b-a160-84e5251cb95e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:05.422425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3648242a45af45adbea4176139c6edec', 'request_id': 'req-65771807-28b3-432b-a160-84e5251cb95e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:05.431841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3648242a45af45adbea4176139c6edec', 'request_id': 'req-65771807-28b3-432b-a160-84e5251cb95e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:05.447849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] The token cache is being invalidate because project 3648242a45af45adbea4176139c6edec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:05.448145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:05.449118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-65771807-28b3-432b-a160-84e5251cb95e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:05.450717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 954/1902] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:05 2022] DELETE /identity/v3/projects/3648242a45af45adbea4176139c6edec => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:05.456465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.457380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.457380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.485866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:05.487339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:05.592525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 949/1903] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:05.601463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.601709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.601904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.629449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:05.631306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:05.642527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 955/1904] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:05.646758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:05.659991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99c0e4-da97-44b2-85dc-ca00f0efc999 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:05.666003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:05.666734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.667003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.667307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.668095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:05.668414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:05.670854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:05.688039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 950/1905] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:05.694548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:05.714875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01556cb1-87e4-4b8a-a9e9-449bc503bffc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:05.722905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:05.723777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.724305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.724890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.726789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:05.730691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:05.769001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 956/1906] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:05 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:05.772352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:05.783711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76026706-54bd-4179-9dce-1c758c07c61f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:05.869883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:05.870583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.870972 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.871331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.872114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:05.874931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:05.904103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 951/1907] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:05 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:05.910110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:05.923416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2603a-f889-4794-af47-f11be3bc194b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:05.928438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:05.929107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.929387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.929660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.930370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:05.932038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:05.943339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 957/1908] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:05.947563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:05.983862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6fbbc-3c96-4509-b6e6-b0b89f359eeb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:05.992677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502587dd-835d-47d4-b5b2-a1bbedef1e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:05.994383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502587dd-835d-47d4-b5b2-a1bbedef1e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:05.994878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502587dd-835d-47d4-b5b2-a1bbedef1e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:05.995308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502587dd-835d-47d4-b5b2-a1bbedef1e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:05.996218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502587dd-835d-47d4-b5b2-a1bbedef1e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:05.999770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502587dd-835d-47d4-b5b2-a1bbedef1e5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:06.033308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 952/1909] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:06.037084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:06.053714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98427939-8850-4109-9832-55488b9806a7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:06.059595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:06.060313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:06.060596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:06.061520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] PATH_INFO: `/v3/projects/6a42fbd327a94c90ae28cad9875ec573/users/c8a63da99b0146cf8d1eabe356de9c7f/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:06.090671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a42fbd327a94c90ae28cad9875ec573, user_id=c8a63da99b0146cf8d1eabe356de9c7f, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:06.096181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:06.130005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 958/1910] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:06 2022] PUT /identity/v3/projects/6a42fbd327a94c90ae28cad9875ec573/users/c8a63da99b0146cf8d1eabe356de9c7f/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:06.142743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:06.143014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:06.143288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:06.182257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:06.194810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] MFA Rules not processed for user `c8a63da99b0146cf8d1eabe356de9c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:06.196135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:06.333375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 953/1911] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:06.335171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:06.461901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd9338-6230-414f-8e00-8b905208a1cf admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:06.473700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:06.474789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:06.475266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:06.475713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:06.478224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:06.486253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:06.540546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 959/1912] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:06 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:06.541279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:06.563572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a122da-a06e-4eba-a084-bf4192f684ba None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:06.675852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:06.677108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:06.677754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:06.678335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:06.679580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:06.683192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:06.725571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 954/1913] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:06 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:06.731241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:06.764509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bbdfb-cbc1-44b1-b95a-ada854c1dc4a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:06.789264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:06.792119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:06.794689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:06.796299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:06.802752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:06.815853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:06.867267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 960/1914] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:06.872300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:06.910258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a5fd9-d034-4fda-9680-a25b0b911cd2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:06.939651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:06.940448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:06.943243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:06.945328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] PATH_INFO: `/v3/projects/e3ddce25008b4b14b5a2ea2308a60084/users/74b9aeb71faf49bd9cd81e4e47f926d4/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.002939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:07.003948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ddce25008b4b14b5a2ea2308a60084, user_id=74b9aeb71faf49bd9cd81e4e47f926d4, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:07.006305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:07.017263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:07.055199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 955/1915] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:06 2022] PUT /identity/v3/projects/e3ddce25008b4b14b5a2ea2308a60084/users/74b9aeb71faf49bd9cd81e4e47f926d4/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:07.058407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18985bba-300f-4471-9c28-97281aa7fcb1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:07.061516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:07.104080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:07.261366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:07.265835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.267570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.270407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] PATH_INFO: `/v3/users/fd6ab8a404414dcb89eff2145e46c19c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.274312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6ab8a404414dcb89eff2145e46c19c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:07.280061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:07.280747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af2a19-cb84-47c3-8755-3ef53c77a0c1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:07.290873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:07.292350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.293187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.293733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.294915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:07.301572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:07.317414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:07.347164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 956/1916] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:07.359399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:07.383822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1497ee-c50f-4740-a2ed-a3159eb39744 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:07.402099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954809b-6cf8-4b84-8ba5-2bd165389be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:07.405191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954809b-6cf8-4b84-8ba5-2bd165389be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.405652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954809b-6cf8-4b84-8ba5-2bd165389be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.409704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954809b-6cf8-4b84-8ba5-2bd165389be7 admin admin] PATH_INFO: `/v3/projects/e3ddce25008b4b14b5a2ea2308a60084/users/74b9aeb71faf49bd9cd81e4e47f926d4/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.415836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6ab8a404414dcb89eff2145e46c19c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:07.417893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954809b-6cf8-4b84-8ba5-2bd165389be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ddce25008b4b14b5a2ea2308a60084, user_id=74b9aeb71faf49bd9cd81e4e47f926d4, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:07.418547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6ab8a404414dcb89eff2145e46c19c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:07.424758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954809b-6cf8-4b84-8ba5-2bd165389be7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:07.452010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6ab8a404414dcb89eff2145e46c19c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:07.452846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 957/1917] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:07 2022] PUT /identity/v3/projects/e3ddce25008b4b14b5a2ea2308a60084/users/74b9aeb71faf49bd9cd81e4e47f926d4/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:07.474654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.474953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.475210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.480134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6ab8a404414dcb89eff2145e46c19c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:07.501825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7050c7a6-ebaa-4e2d-82ae-f28b2ead8a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6ab8a404414dcb89eff2145e46c19c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:07.555149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] MFA Rules not processed for user `74b9aeb71faf49bd9cd81e4e47f926d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:07.557568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:07.561776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 961/1918] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:07 2022] DELETE /identity/v3/users/fd6ab8a404414dcb89eff2145e46c19c => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:07.563574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.563865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.564156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.605396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:07.609249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:07.665560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 958/1919] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:07.696359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:07.698239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:07.767324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 962/1920] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:07.794429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:07.800333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:07.885301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00314747-f199-4f08-b803-22cffcce9e14 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:07.897182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:07.897843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.898131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.898407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.900570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:07.913817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:07.947916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6cc1b-84b7-4418-9c75-7fcb36b39a94 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:07.956226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:07.957379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:07.957874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:07.958370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:07.978241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:07.981046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:07.981251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:07.983126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:07.994067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 959/1921] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:08.001701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 963/1922] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:08.553485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:08.555353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:08.603828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e6cb7-6c74-4e72-9d26-9bb6c96fb841 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:08.626264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:08.627396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:08.627877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:08.628380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] PATH_INFO: `/v3/projects/c15c132af6024326a0ef763397b66d44` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:08.631796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15c132af6024326a0ef763397b66d44)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:08.646292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:08.720627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:08.798237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15c132af6024326a0ef763397b66d44', 'request_id': 'req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:08.829365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15c132af6024326a0ef763397b66d44', 'request_id': 'req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:08.850356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15c132af6024326a0ef763397b66d44', 'request_id': 'req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:08.886247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] The token cache is being invalidate because project c15c132af6024326a0ef763397b66d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:08.886419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:08.887294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:08.893226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 960/1923] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:08 2022] DELETE /identity/v3/projects/c15c132af6024326a0ef763397b66d44 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:08.894761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:08.917461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4054aff-3ad0-437d-ab6f-fb0a78da6f66 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:09.059535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:09.060203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:09.060487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:09.060833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] PATH_INFO: `/v3/users/5dca86732e7447e38501b266958cc8a1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:09.062689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dca86732e7447e38501b266958cc8a1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:09.065102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:09.187333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dca86732e7447e38501b266958cc8a1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:09.188789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dca86732e7447e38501b266958cc8a1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:09.225914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dca86732e7447e38501b266958cc8a1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:09.238335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dca86732e7447e38501b266958cc8a1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:09.249044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dca86732e7447e38501b266958cc8a1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:09.256092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 964/1924] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:08 2022] DELETE /identity/v3/users/5dca86732e7447e38501b266958cc8a1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:09.954470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:09.957381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:09.976206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be2c9b-79ec-4b00-83b4-0e82b9a647d4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:10.130256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:10.131232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:10.133298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:10.133872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] PATH_INFO: `/v3/projects/f2920cb8a4c74818bd8be9394bf4fd6f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:10.135715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2920cb8a4c74818bd8be9394bf4fd6f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:10.141024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:10.252145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2920cb8a4c74818bd8be9394bf4fd6f', 'request_id': 'req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.263891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2920cb8a4c74818bd8be9394bf4fd6f', 'request_id': 'req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.277379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2920cb8a4c74818bd8be9394bf4fd6f', 'request_id': 'req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.325858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] The token cache is being invalidate because project f2920cb8a4c74818bd8be9394bf4fd6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:10.327336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.328668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.332106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 961/1925] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:09 2022] DELETE /identity/v3/projects/f2920cb8a4c74818bd8be9394bf4fd6f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:10.338336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:10.340878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:10.376393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b41fb7-ab15-4e4e-8f0c-b3948dbc73d4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:10.487644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:10.488469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:10.488780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:10.489174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] PATH_INFO: `/v3/users/e2472be472a14070814e7294fb9b717d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:10.490717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2472be472a14070814e7294fb9b717d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:10.492920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:10.497166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:10.591609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2472be472a14070814e7294fb9b717d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.592846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2472be472a14070814e7294fb9b717d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.611545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2472be472a14070814e7294fb9b717d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.626699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2472be472a14070814e7294fb9b717d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.636628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-20b72f1b-b62a-4b73-a41c-d6e4a54c10e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2472be472a14070814e7294fb9b717d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:10.648685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 965/1926] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:10 2022] DELETE /identity/v3/users/e2472be472a14070814e7294fb9b717d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:11.407488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:11.411154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:11.442598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08501b-6c6e-422f-ad8b-17ec2f2d172f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:11.566217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:11.606386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:11.607064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:11.607342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:11.607749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] PATH_INFO: `/v3/projects/0109633806ec473fb2eb7feb9ce7859b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:11.609374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] RBAC: Authorizing `identity:delete_project(project_id=0109633806ec473fb2eb7feb9ce7859b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:11.611602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:11.685499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0109633806ec473fb2eb7feb9ce7859b', 'request_id': 'req-7b778f96-0800-4151-8db4-aaf3dee192db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:11.693476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0109633806ec473fb2eb7feb9ce7859b', 'request_id': 'req-7b778f96-0800-4151-8db4-aaf3dee192db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:11.705021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0109633806ec473fb2eb7feb9ce7859b', 'request_id': 'req-7b778f96-0800-4151-8db4-aaf3dee192db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:11.735866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] The token cache is being invalidate because project 0109633806ec473fb2eb7feb9ce7859b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:11.736691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:11.738880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7b778f96-0800-4151-8db4-aaf3dee192db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:11.765451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 962/1927] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:11 2022] DELETE /identity/v3/projects/0109633806ec473fb2eb7feb9ce7859b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:11.774328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:11.775097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:11.775556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:11.814622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:11.818501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:11.966711 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 966/1928] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:11.977275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:11.977943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:11.978382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.039142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:12.041901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:12.057057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 963/1929] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:12.061876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.083813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0cb04-f256-425f-8a4b-f4ccdf584617 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.090122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.090832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.091116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.091395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.092163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:12.092475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.094180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.109818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 967/1930] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:12.115764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.134212 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c61d7-8ad0-466c-9a99-ffcd706ede8b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.152470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.157101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.157613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.158109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.159428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.168852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.263785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 964/1931] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:12 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:12.268477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.285592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ee3b0-36d3-4251-a749-157edbc0767d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.297310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.299263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:12.496550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856318c-2b73-425a-aceb-934df1d61768 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.503552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.505576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.505576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.505979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.506227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.506656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.506937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.507223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] PATH_INFO: `/v3/users/c8a63da99b0146cf8d1eabe356de9c7f` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.507848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.510272 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.517944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a63da99b0146cf8d1eabe356de9c7f)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.519956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.556091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 968/1932] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:12 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:12.556993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:12.560079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.579901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682c8df-0643-4777-9f7f-7c5b8dc34f17 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.604673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.605362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.605637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.608802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.614248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.617223 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.629487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:12.633395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a63da99b0146cf8d1eabe356de9c7f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:12.644704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 969/1933] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:12.648639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.651425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a63da99b0146cf8d1eabe356de9c7f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:12.662735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a63da99b0146cf8d1eabe356de9c7f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:12.663232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0ca0c-f459-41e1-a84a-259c68cb4796 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.664003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a63da99b0146cf8d1eabe356de9c7f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:12.664952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b4f65508-0627-4c79-a60c-366f6d7968dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a63da99b0146cf8d1eabe356de9c7f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:12.670555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.671643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.672235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.672607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.673830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.688365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.691795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 965/1934] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:12 2022] DELETE /identity/v3/users/c8a63da99b0146cf8d1eabe356de9c7f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:12.698385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.698761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.699111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.721856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 970/1935] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:12.722696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:12.724140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:12.726987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.730224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:12.760892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7049-7af9-4b17-a7fc-392eea5189f8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:12.848227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a93890-98ab-4f48-ba16-10ecca4080a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:12.849409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a93890-98ab-4f48-ba16-10ecca4080a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.849409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a93890-98ab-4f48-ba16-10ecca4080a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.849740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a93890-98ab-4f48-ba16-10ecca4080a6 admin admin] PATH_INFO: `/v3/projects/61287d24823b4eab9cd2ad38fcea2072/users/643125de6092408db06e345781d89680/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:12.855490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 966/1936] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:12.873914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:12.878421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a93890-98ab-4f48-ba16-10ecca4080a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61287d24823b4eab9cd2ad38fcea2072, user_id=643125de6092408db06e345781d89680, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:12.881240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:12.881729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a93890-98ab-4f48-ba16-10ecca4080a6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:12.911270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 971/1937] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:12 2022] PUT /identity/v3/projects/61287d24823b4eab9cd2ad38fcea2072/users/643125de6092408db06e345781d89680/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:12.929673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:12.930284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:12.930625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:13.014979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] MFA Rules not processed for user `643125de6092408db06e345781d89680`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:13.016518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:13.085714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87ce4e-fe96-4015-a951-b3c596f1f0d3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:13.245520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:13.247522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:13.250734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 972/1938] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:13.251407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:13.251861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:13.257170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:13.300416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:13.354890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:13.356783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:13.386190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 967/1939] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:13.414660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:13.461101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ecd79-77a0-4b4b-9ce1-8dcbff3dda79 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:13.468180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:13.469495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:13.469989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:13.470398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:13.471334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:13.474411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:13.539141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 973/1940] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:13 2022] POST /identity/v3/projects => generated 385 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:13.541384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:13.561819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4575f-7532-40dc-bbb1-5d1e7eae7310 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:13.722365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:13.723020 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:13.723304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:13.723577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:13.724260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:13.727005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:13.780317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 968/1941] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:13 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:13.782271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:13.829596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce21f3-b05b-4f74-8ded-3a04f7bfa491 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:13.852289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:13.854620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:13.855409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:13.856123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:13.865379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:13.868834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:13.897113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 974/1942] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:13.898516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:13.916161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcbd73-2c3f-4dd4-9f94-682bcccbebb8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:13.926180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc34cb-d2ee-468c-81a3-ad50ceb88060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:13.927077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc34cb-d2ee-468c-81a3-ad50ceb88060 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:13.927574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc34cb-d2ee-468c-81a3-ad50ceb88060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:13.928173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc34cb-d2ee-468c-81a3-ad50ceb88060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:13.929559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc34cb-d2ee-468c-81a3-ad50ceb88060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:13.933345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc34cb-d2ee-468c-81a3-ad50ceb88060 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:13.954295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:13.961276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 969/1943] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:13.968707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9c8ed-caa2-4d3a-a2aa-3b757081795d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:13.974631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:13.975676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:13.976188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:13.976481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] PATH_INFO: `/v3/projects/d6978c821a3e4cc7bce9695232f7ca61/users/80fea8248013435cba9e5fed4a0e7466/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.020068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6978c821a3e4cc7bce9695232f7ca61, user_id=80fea8248013435cba9e5fed4a0e7466, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.025522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.056703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 975/1944] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:13 2022] PUT /identity/v3/projects/d6978c821a3e4cc7bce9695232f7ca61/users/80fea8248013435cba9e5fed4a0e7466/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:14.070553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.070553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.070553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.097925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.100098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:14.124164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] MFA Rules not processed for user `80fea8248013435cba9e5fed4a0e7466`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:14.125921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:14.242780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b748bdd-5a91-499d-9bfe-a1e5ee9c7e05 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.246719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 970/1945] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:14.247972 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.248640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.249487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.249776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.250054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] PATH_INFO: `/v3/projects/6a42fbd327a94c90ae28cad9875ec573` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.251331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a42fbd327a94c90ae28cad9875ec573)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.253871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.262737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385aec25-931a-4058-ba80-a510b71e3e90 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.270237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.271443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.271797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.272316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.273443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.276478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.333132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a42fbd327a94c90ae28cad9875ec573', 'request_id': 'req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.343278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 971/1946] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:14 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:14.348059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.356607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a42fbd327a94c90ae28cad9875ec573', 'request_id': 'req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.377978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a42fbd327a94c90ae28cad9875ec573', 'request_id': 'req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.382859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:14.424409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] The token cache is being invalidate because project 6a42fbd327a94c90ae28cad9875ec573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:14.425114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.426936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.430165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 976/1947] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:14 2022] DELETE /identity/v3/projects/6a42fbd327a94c90ae28cad9875ec573 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:14.436949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.441324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:14.507443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69beb-7cf0-4b24-bb5f-302374a91432 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.515835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.515835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.515835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.515835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.516644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.518765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.555034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 972/1948] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:14 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:14.558070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.562104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:14.589248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:14.609367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83498be3-ad24-4838-a4b9-77fc8c901d02 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.620571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.625140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.625476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.625676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.628297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.631351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.660526 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9556dd-8ebe-4b17-b9ca-ed9acfcb2d54 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.663293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 973/1949] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:14.666606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.667377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.667722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.667840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.668112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] PATH_INFO: `/v3/users/74b9aeb71faf49bd9cd81e4e47f926d4` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.669639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74b9aeb71faf49bd9cd81e4e47f926d4)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.673163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.682101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120583a-b162-42cb-ac58-4646ab66dfbd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.688158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.688864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.689203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.689483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] PATH_INFO: `/v3/projects/cf389a7075074e6388593aaeb719143d/users/422ae20c2e4644ecb91a3a5eaa3dd470/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.722606 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf389a7075074e6388593aaeb719143d, user_id=422ae20c2e4644ecb91a3a5eaa3dd470, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.728455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.740666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74b9aeb71faf49bd9cd81e4e47f926d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.742309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74b9aeb71faf49bd9cd81e4e47f926d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.750540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 974/1950] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:14 2022] PUT /identity/v3/projects/cf389a7075074e6388593aaeb719143d/users/422ae20c2e4644ecb91a3a5eaa3dd470/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:14.755615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.758615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:14.762366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74b9aeb71faf49bd9cd81e4e47f926d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.772596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74b9aeb71faf49bd9cd81e4e47f926d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.782555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74b9aeb71faf49bd9cd81e4e47f926d4'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:14.789687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 977/1951] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:14 2022] DELETE /identity/v3/users/74b9aeb71faf49bd9cd81e4e47f926d4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:14.866992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000f7bf-8f86-46f2-b0d3-a7370773de13 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:14.961970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96d020-964c-4634-b230-81d74c8b25b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:14.962589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d020-964c-4634-b230-81d74c8b25b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:14.962954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d020-964c-4634-b230-81d74c8b25b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:14.963846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96d020-964c-4634-b230-81d74c8b25b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:14.964113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d020-964c-4634-b230-81d74c8b25b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:14.965939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f96d020-964c-4634-b230-81d74c8b25b0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:14.978237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 975/1952] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:14.985425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:14.999565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c382350-36f4-4278-95ff-422181d71345 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:15.005939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:15.007277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.007921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.008407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] PATH_INFO: `/v3/projects/cf389a7075074e6388593aaeb719143d/users/422ae20c2e4644ecb91a3a5eaa3dd470/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.011827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf389a7075074e6388593aaeb719143d, user_id=422ae20c2e4644ecb91a3a5eaa3dd470, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:15.015980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:15.036013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 978/1953] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:14 2022] PUT /identity/v3/projects/cf389a7075074e6388593aaeb719143d/users/422ae20c2e4644ecb91a3a5eaa3dd470/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:15.051688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.052654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.053862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.099527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:15.121892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] MFA Rules not processed for user `422ae20c2e4644ecb91a3a5eaa3dd470`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:15.125742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:15.298300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:15.304456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:15.323924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 976/1954] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:15.356466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:15.361975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:15.436046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85434f4-a9fb-4a34-825c-15c4a9863cd6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:15.446681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:15.447990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.448492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.449371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] PATH_INFO: `/v3/projects/e3ddce25008b4b14b5a2ea2308a60084` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.452120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ddce25008b4b14b5a2ea2308a60084)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:15.455693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:15.469389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edcfb-b91e-4528-85a0-86cd47cb6b45 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:15.476082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:15.476905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.476905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.477415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.486057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:15.527232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:15.540782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ddce25008b4b14b5a2ea2308a60084', 'request_id': 'req-af157c1c-e1fe-4c6a-85ba-4904362916ad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:15.554401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ddce25008b4b14b5a2ea2308a60084', 'request_id': 'req-af157c1c-e1fe-4c6a-85ba-4904362916ad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:15.564935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ddce25008b4b14b5a2ea2308a60084', 'request_id': 'req-af157c1c-e1fe-4c6a-85ba-4904362916ad'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:15.580476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:15.582024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d754e-4c5c-4e7c-b151-f2e39f52e536 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:15.583318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] The token cache is being invalidate because project e3ddce25008b4b14b5a2ea2308a60084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:15.583811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:15.585355 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-af157c1c-e1fe-4c6a-85ba-4904362916ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:15.588117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 979/1955] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:15 2022] DELETE /identity/v3/projects/e3ddce25008b4b14b5a2ea2308a60084 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:15.594576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.595048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.595529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.595794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 977/1956] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:15.646648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:15.649480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:15.776610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 980/1957] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:15.787915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.788439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.788953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.839029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:15.839609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:15.850347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 978/1958] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:15.853639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:15.864910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478f8a-7de1-403e-9b27-72d53554d774 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:15.871325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:15.871971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.872257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.872534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.873924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:15.874356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:15.876048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:15.892049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 981/1959] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:15.896571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:15.912925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e07c18-731e-48e8-be99-15099bea42c3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:15.930098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:15.932735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:15.933842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:15.934567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:15.937267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:15.941506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:15.994822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 979/1960] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:15 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:16.001266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:16.039964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7756fc-37c7-43e4-bbe4-b226614df0cc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:16.277935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:16.279112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:16.281412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:16.281728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:16.284704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:16.288486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:16.345812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 982/1961] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:15 2022] POST /identity/v3/users => generated 288 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:16.351461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:16.374335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e192ab-f91e-4b4b-8993-372cc61db416 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:16.380612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:16.381296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:16.381720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:16.382598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:16.382873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:16.384627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:16.398519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 980/1962] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:16.413563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:16.454644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df655b-3748-468e-aca4-9ec6450e435e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:16.483601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497be73-19ba-47a8-aca2-58ed8bd75f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:16.485211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497be73-19ba-47a8-aca2-58ed8bd75f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:16.486187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497be73-19ba-47a8-aca2-58ed8bd75f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:16.487013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497be73-19ba-47a8-aca2-58ed8bd75f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:16.488473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497be73-19ba-47a8-aca2-58ed8bd75f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:16.491289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497be73-19ba-47a8-aca2-58ed8bd75f59 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:16.515847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 983/1963] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:16.522465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:16.533294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f5514-ddca-436a-a660-e05ec41c40ec admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:16.540044 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:16.540757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:16.541224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:16.541500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] PATH_INFO: `/v3/projects/475851022c9844dc93ae9fdc4ed7c8c8/users/149bc9eb495f40e59a645d89b9df20a6/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:16.554317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:16.567019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=475851022c9844dc93ae9fdc4ed7c8c8, user_id=149bc9eb495f40e59a645d89b9df20a6, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:16.570211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:16.591879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 981/1964] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:16 2022] PUT /identity/v3/projects/475851022c9844dc93ae9fdc4ed7c8c8/users/149bc9eb495f40e59a645d89b9df20a6/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:16.610862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:16.611376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:16.611944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:16.673069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] MFA Rules not processed for user `149bc9eb495f40e59a645d89b9df20a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:16.675477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:16.705444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:16.803938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 984/1965] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:16.805728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:16.916238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4ce98-574d-460b-94da-039f2f812fc5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:16.921982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:16.922632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:16.922914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:16.923187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:16.923895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:16.925923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:16.968806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 982/1966] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:16 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:16.970236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:16.983491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc98793-b94f-48ae-a335-0a6e407fff97 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.075440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.077060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.077427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.077864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.078697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.081077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.111072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 985/1967] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:16 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:17.112609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.125833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf415713-2b9b-4149-af50-01560de0992f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.139440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.140312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.140595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.140873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.141632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.143378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.156319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 983/1968] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:17.161491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.174158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366c3b7-ff49-492e-9598-0b0cbcc3a561 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.179884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12c4b7-9e77-4b52-be42-0e2def61a286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.180540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12c4b7-9e77-4b52-be42-0e2def61a286 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.180822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12c4b7-9e77-4b52-be42-0e2def61a286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.181368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12c4b7-9e77-4b52-be42-0e2def61a286 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.182048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12c4b7-9e77-4b52-be42-0e2def61a286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.183781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12c4b7-9e77-4b52-be42-0e2def61a286 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.214104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 986/1969] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:17.227146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.243092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14a59e-b50b-449f-b935-c6862ee9404d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.249340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.250209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.250537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.250954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] PATH_INFO: `/v3/projects/8dc82ca9c17b4530adb86a0353138eb0/users/93ca25890fe84bd9ab1483aefdffe7ef/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.276278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc82ca9c17b4530adb86a0353138eb0, user_id=93ca25890fe84bd9ab1483aefdffe7ef, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.279081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.296211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 984/1970] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:17 2022] PUT /identity/v3/projects/8dc82ca9c17b4530adb86a0353138eb0/users/93ca25890fe84bd9ab1483aefdffe7ef/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:17.315903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.316888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.316888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.369157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] MFA Rules not processed for user `93ca25890fe84bd9ab1483aefdffe7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:17.371606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:17.476068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 987/1971] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:17.480746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.572365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf386203-c66c-4752-8aca-ad5ab4c3590f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.578151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.578862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.579148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.579430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.580238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.582199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.617062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 985/1972] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:17 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:17.618677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.631975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23890086-2abf-4e7a-a7b6-034196d77fd7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.664686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:17.705174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.705814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.706093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.706361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.707038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.708997 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.747032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 988/1973] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:17 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:17.749300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.761018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130ee4a-3a55-40db-b056-5f5621523b47 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.765927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.766566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.766842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.767112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.767940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.769632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.783151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 986/1974] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:17.788244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.808601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076415ef-a071-4a46-92bc-d536882b4fd0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.819094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:17.820257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:17.820759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:17.821278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] PATH_INFO: `/v3/projects/9548ae94106f4833adf1dbd3330ce935/users/f5d6dad0a71e4d2695d4a2ed66fe09d9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:17.844096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9548ae94106f4833adf1dbd3330ce935, user_id=f5d6dad0a71e4d2695d4a2ed66fe09d9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:17.847107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:17.865513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 989/1975] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:17 2022] PUT /identity/v3/projects/9548ae94106f4833adf1dbd3330ce935/users/f5d6dad0a71e4d2695d4a2ed66fe09d9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:17.870794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:17.890218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6f5f7-a012-4e4d-b6ca-0dd0094cfc6b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:17.902337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:17.996551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7bddb-0156-4a9b-840f-1e28e99134eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:18.002080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7bddb-0156-4a9b-840f-1e28e99134eb admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:18.002598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7bddb-0156-4a9b-840f-1e28e99134eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:18.003112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7bddb-0156-4a9b-840f-1e28e99134eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:18.004380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7bddb-0156-4a9b-840f-1e28e99134eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:18.007964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7bddb-0156-4a9b-840f-1e28e99134eb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:18.037128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 987/1976] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:18.044507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:18.064498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f288e-96a1-41c0-a815-2a7fda8e0a30 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:18.075626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:18.077276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:18.077445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:18.077733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] PATH_INFO: `/v3/projects/9548ae94106f4833adf1dbd3330ce935/users/f5d6dad0a71e4d2695d4a2ed66fe09d9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:18.081964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9548ae94106f4833adf1dbd3330ce935, user_id=f5d6dad0a71e4d2695d4a2ed66fe09d9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:18.086978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:18.102876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 990/1977] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:18 2022] PUT /identity/v3/projects/9548ae94106f4833adf1dbd3330ce935/users/f5d6dad0a71e4d2695d4a2ed66fe09d9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:18.115754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:18.116419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:18.117067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:18.176605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] MFA Rules not processed for user `f5d6dad0a71e4d2695d4a2ed66fe09d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:18.178051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:18.329271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 988/1978] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:18.343875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:18.349719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:18.466608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830446a-8c53-4ccd-b538-90efc2353dbc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:18.471966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:18.472975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:18.472975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:18.473182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:18.553750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:18.555384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:18.568865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 991/1979] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:20.890794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:20.892622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:20.905721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af84a5-f117-4595-9c52-ca51c4d1bd9a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:20.910137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:20.910783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:20.911060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:20.911335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:20.913230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:20.988479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:20.989990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:21.001380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 989/1980] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:23.283798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:23.294662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf278232-8eb6-4e47-a404-8c5ada3aa557 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:23.303494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:23.304187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:23.304472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:23.304747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:23.306675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:23.348000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:23.384825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:23.386626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:23.401811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 992/1981] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:24.558554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:24.567999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10527901-2f3b-4658-be6c-cd254024fe30 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:24.572117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:24.572780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:24.573099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:24.573376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:24.636751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:24.694169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:24.696087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:24.714132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 990/1982] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:26.360181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:26.392359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33a8bb-0a43-42ca-bfb9-4e0c4b7f3d85 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:26.401374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39ab75-bdac-4a12-9e57-e0aed710466a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:26.402708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39ab75-bdac-4a12-9e57-e0aed710466a service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:26.403388 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39ab75-bdac-4a12-9e57-e0aed710466a service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:26.404070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39ab75-bdac-4a12-9e57-e0aed710466a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:26.430637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39ab75-bdac-4a12-9e57-e0aed710466a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:26.434274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39ab75-bdac-4a12-9e57-e0aed710466a service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:26.457368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 993/1983] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:27.936550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:27.937913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:28.058397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69886883-0d3a-4d25-9f55-57573fe0ba3e service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:28.066929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:28.068689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:28.069252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:28.070152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] PATH_INFO: `/v3/users/643125de6092408db06e345781d89680` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:28.073076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=643125de6092408db06e345781d89680)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:28.075329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:28.157239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643125de6092408db06e345781d89680'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:28.172236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643125de6092408db06e345781d89680'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:28.187636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643125de6092408db06e345781d89680'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:28.188384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643125de6092408db06e345781d89680'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:28.189491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643125de6092408db06e345781d89680'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:28.231186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 991/1984] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:27 2022] DELETE /identity/v3/users/643125de6092408db06e345781d89680 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:28.237801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:28.237801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:28.237801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:28.266594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:28.267911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:28.408995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 994/1985] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:28.425284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:28.425284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:28.574562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:28.583200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a922843-70e6-4c79-a51f-8c88921c81cd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:28.594859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:28.596133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:28.596357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:28.596725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:28.613623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:28.616391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:28.637828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 992/1986] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:29.104395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:29.106589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:29.130330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b4c4c-80e3-4914-828f-d5503cc128c0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:29.144719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:29.146296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:29.146784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:29.147606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] PATH_INFO: `/v3/projects/61287d24823b4eab9cd2ad38fcea2072` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:29.149924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61287d24823b4eab9cd2ad38fcea2072)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:29.153352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:29.222279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61287d24823b4eab9cd2ad38fcea2072', 'request_id': 'req-77e04fda-962a-4c53-a55e-8ed3fda48a1f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.231925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61287d24823b4eab9cd2ad38fcea2072', 'request_id': 'req-77e04fda-962a-4c53-a55e-8ed3fda48a1f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.241408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61287d24823b4eab9cd2ad38fcea2072', 'request_id': 'req-77e04fda-962a-4c53-a55e-8ed3fda48a1f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.255862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] The token cache is being invalidate because project 61287d24823b4eab9cd2ad38fcea2072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:29.256304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.257328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.259071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 995/1987] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:29 2022] DELETE /identity/v3/projects/61287d24823b4eab9cd2ad38fcea2072 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:29.265559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:29.277697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009c0cd-7c92-43b0-a2e8-5107bad38188 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:29.355607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:29.355607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:29.355607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:29.355607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] PATH_INFO: `/v3/users/80fea8248013435cba9e5fed4a0e7466` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:29.356153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80fea8248013435cba9e5fed4a0e7466)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:29.356153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:29.427693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80fea8248013435cba9e5fed4a0e7466'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.436321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80fea8248013435cba9e5fed4a0e7466'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.446888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80fea8248013435cba9e5fed4a0e7466'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.447296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80fea8248013435cba9e5fed4a0e7466'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.447924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80fea8248013435cba9e5fed4a0e7466'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:29.470353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 993/1988] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:29 2022] DELETE /identity/v3/users/80fea8248013435cba9e5fed4a0e7466 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:29.871277 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:29.875917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:29.911285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e04fda-962a-4c53-a55e-8ed3fda48a1f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:29.982900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:30.023719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:30.024445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:30.024770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:30.025083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] PATH_INFO: `/v3/projects/d6978c821a3e4cc7bce9695232f7ca61` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:30.027250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6978c821a3e4cc7bce9695232f7ca61)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:30.030150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:30.133276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6978c821a3e4cc7bce9695232f7ca61', 'request_id': 'req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.151851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6978c821a3e4cc7bce9695232f7ca61', 'request_id': 'req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.169358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6978c821a3e4cc7bce9695232f7ca61', 'request_id': 'req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.189203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] The token cache is being invalidate because project d6978c821a3e4cc7bce9695232f7ca61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:30.189927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.191393 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.193927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 996/1989] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:29 2022] DELETE /identity/v3/projects/d6978c821a3e4cc7bce9695232f7ca61 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:30.199583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:30.204510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:30.231465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4914d48f-8d15-46ec-be7a-c6eb3b102d4b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:30.357596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:30.360203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:30.360203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:30.360855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] PATH_INFO: `/v3/users/422ae20c2e4644ecb91a3a5eaa3dd470` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:30.364361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=422ae20c2e4644ecb91a3a5eaa3dd470)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:30.368653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:30.403995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:30.495568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422ae20c2e4644ecb91a3a5eaa3dd470'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.507072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422ae20c2e4644ecb91a3a5eaa3dd470'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.521636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422ae20c2e4644ecb91a3a5eaa3dd470'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.522401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422ae20c2e4644ecb91a3a5eaa3dd470'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.523178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1a3adaa8-b4cf-4cd5-a549-faea448c6e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422ae20c2e4644ecb91a3a5eaa3dd470'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.562255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 994/1990] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:30 2022] DELETE /identity/v3/users/422ae20c2e4644ecb91a3a5eaa3dd470 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:30.662491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:30.665161 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:30.688802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a4c5-9b9b-4fb9-9dda-cac218d32160 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:30.805886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:30.806581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:30.806857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:30.807128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] PATH_INFO: `/v3/users/149bc9eb495f40e59a645d89b9df20a6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:30.808508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149bc9eb495f40e59a645d89b9df20a6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:30.810726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:30.888184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149bc9eb495f40e59a645d89b9df20a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.890054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149bc9eb495f40e59a645d89b9df20a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.905789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149bc9eb495f40e59a645d89b9df20a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.914915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149bc9eb495f40e59a645d89b9df20a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.916947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:30.928303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149bc9eb495f40e59a645d89b9df20a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:30.952405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 997/1991] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:30 2022] DELETE /identity/v3/users/149bc9eb495f40e59a645d89b9df20a6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:30.953784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:30.954640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:30.955022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:30.960554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:30.962885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:30.988450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:30.990008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:31.065198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123832c7-d406-4406-8fa0-d8b8ad7d4506 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:31.070614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:31.071564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:31.072033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:31.072303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] PATH_INFO: `/v3/projects/cf389a7075074e6388593aaeb719143d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:31.073625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf389a7075074e6388593aaeb719143d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:31.075446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:31.085587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 995/1992] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:31.099169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:31.101339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:31.132192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf389a7075074e6388593aaeb719143d', 'request_id': 'req-b2ca6bca-28ac-4997-a64a-26637388cac3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:31.144396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf389a7075074e6388593aaeb719143d', 'request_id': 'req-b2ca6bca-28ac-4997-a64a-26637388cac3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:31.152440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf389a7075074e6388593aaeb719143d', 'request_id': 'req-b2ca6bca-28ac-4997-a64a-26637388cac3'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:31.166987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] The token cache is being invalidate because project cf389a7075074e6388593aaeb719143d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:31.167257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:31.168078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:31.169656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 998/1993] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:30 2022] DELETE /identity/v3/projects/cf389a7075074e6388593aaeb719143d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:31.187922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc11f7-0d20-440b-aaea-8c12866645ed None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:31.277250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:31.286113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:31.286939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:31.287227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:31.288684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:31.291980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:31.407588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:31.409786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:31.431672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 996/1994] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:31.869253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:31.871088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:31.889190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca6bca-28ac-4997-a64a-26637388cac3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:31.900795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:31.901799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:31.902245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:31.902678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] PATH_INFO: `/v3/projects/475851022c9844dc93ae9fdc4ed7c8c8` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:31.904992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] RBAC: Authorizing `identity:delete_project(project_id=475851022c9844dc93ae9fdc4ed7c8c8)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:31.908323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:32.012625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '475851022c9844dc93ae9fdc4ed7c8c8', 'request_id': 'req-5d83466c-54f0-4470-8d66-b6341d29258d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.025100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '475851022c9844dc93ae9fdc4ed7c8c8', 'request_id': 'req-5d83466c-54f0-4470-8d66-b6341d29258d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.034566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '475851022c9844dc93ae9fdc4ed7c8c8', 'request_id': 'req-5d83466c-54f0-4470-8d66-b6341d29258d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.052826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] The token cache is being invalidate because project 475851022c9844dc93ae9fdc4ed7c8c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:32.053533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.054571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.056667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 999/1995] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:31 2022] DELETE /identity/v3/projects/475851022c9844dc93ae9fdc4ed7c8c8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:32.062384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:32.077958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d110e-7a53-4218-afb6-d9d1bc2953ee service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:32.172545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:32.173598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:32.173890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:32.174172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] PATH_INFO: `/v3/users/93ca25890fe84bd9ab1483aefdffe7ef` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:32.175479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ca25890fe84bd9ab1483aefdffe7ef)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:32.177520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:32.243001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ca25890fe84bd9ab1483aefdffe7ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.252774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ca25890fe84bd9ab1483aefdffe7ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.263203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ca25890fe84bd9ab1483aefdffe7ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.264485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ca25890fe84bd9ab1483aefdffe7ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.265270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ca25890fe84bd9ab1483aefdffe7ef'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.289006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 997/1996] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:32 2022] DELETE /identity/v3/users/93ca25890fe84bd9ab1483aefdffe7ef => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:32.785906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:32.787608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:32.804296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:32.815321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83466c-54f0-4470-8d66-b6341d29258d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:32.904251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:32.905148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:32.905305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:32.905548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] PATH_INFO: `/v3/projects/8dc82ca9c17b4530adb86a0353138eb0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:32.907494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc82ca9c17b4530adb86a0353138eb0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:32.909540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:32.975222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc82ca9c17b4530adb86a0353138eb0', 'request_id': 'req-2092e659-957e-475b-810b-2e29cbca3cbc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.987354 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc82ca9c17b4530adb86a0353138eb0', 'request_id': 'req-2092e659-957e-475b-810b-2e29cbca3cbc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:32.995417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc82ca9c17b4530adb86a0353138eb0', 'request_id': 'req-2092e659-957e-475b-810b-2e29cbca3cbc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.018765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] The token cache is being invalidate because project 8dc82ca9c17b4530adb86a0353138eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:33.019153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.020041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.022392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1000/1997] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:32 2022] DELETE /identity/v3/projects/8dc82ca9c17b4530adb86a0353138eb0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:33.026568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:33.028234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:33.042496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831cdae-5817-4cd4-b873-249c5736aec2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:33.057677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:33.112253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:33.112921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:33.113307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:33.113585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] PATH_INFO: `/v3/users/f5d6dad0a71e4d2695d4a2ed66fe09d9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:33.114800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d6dad0a71e4d2695d4a2ed66fe09d9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:33.116775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:33.176713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d6dad0a71e4d2695d4a2ed66fe09d9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.186114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d6dad0a71e4d2695d4a2ed66fe09d9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.196555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d6dad0a71e4d2695d4a2ed66fe09d9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.197258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d6dad0a71e4d2695d4a2ed66fe09d9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.198162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20397c61-875e-446b-9cca-65cefbbfcc22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d6dad0a71e4d2695d4a2ed66fe09d9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.225617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 998/1998] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:33 2022] DELETE /identity/v3/users/f5d6dad0a71e4d2695d4a2ed66fe09d9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:33.568520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:33.570542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:33.587004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092e659-957e-475b-810b-2e29cbca3cbc admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:33.671376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:33.672197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:33.672489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:33.672918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] PATH_INFO: `/v3/projects/9548ae94106f4833adf1dbd3330ce935` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:33.674235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9548ae94106f4833adf1dbd3330ce935)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:33.676472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:33.728389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:33.746144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9548ae94106f4833adf1dbd3330ce935', 'request_id': 'req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.756043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9548ae94106f4833adf1dbd3330ce935', 'request_id': 'req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.764633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9548ae94106f4833adf1dbd3330ce935', 'request_id': 'req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.782374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] The token cache is being invalidate because project 9548ae94106f4833adf1dbd3330ce935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:33.782773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.783610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b1452651-93b6-4d1d-83a5-ce3ca7de65dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:33.789845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1001/1999] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:33 2022] DELETE /identity/v3/projects/9548ae94106f4833adf1dbd3330ce935 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:33.794636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:33.795471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:33.795763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:33.826941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:33.828352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:33.909399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 999/2000] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:33.918719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759536df-f788-4765-976c-a21f0cf732ca None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:33.919253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759536df-f788-4765-976c-a21f0cf732ca None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:33.919444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759536df-f788-4765-976c-a21f0cf732ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:33.949462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-759536df-f788-4765-976c-a21f0cf732ca None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:33.952101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-759536df-f788-4765-976c-a21f0cf732ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:33.968343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1002/2001] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:33.972765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:33.983798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc437517-aef2-4131-aa79-3c5ede655c81 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:33.993473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:33.994270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:33.994640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:33.995288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:33.996115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:33.996605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:33.999922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.014953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1000/2002] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:34.020064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759536df-f788-4765-976c-a21f0cf732ca None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.030971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759536df-f788-4765-976c-a21f0cf732ca None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.037485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.038136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.038579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.039057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.040078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.044377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.110027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1003/2003] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:34 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:34.114425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.134366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7ed74-3798-4b05-9dcc-1cd398d0e347 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.183910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:34.273797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95ef93-1e50-439c-9310-eddd8f06319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.274490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95ef93-1e50-439c-9310-eddd8f06319d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.275238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95ef93-1e50-439c-9310-eddd8f06319d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.275583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95ef93-1e50-439c-9310-eddd8f06319d admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.277045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95ef93-1e50-439c-9310-eddd8f06319d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.278804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95ef93-1e50-439c-9310-eddd8f06319d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.323026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1001/2004] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:34 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:34.327902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.354834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf53653-a3c5-4dbd-a19a-868107a764ab admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.373999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a755dd1-a5d1-45e7-a248-5f10c69e7e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.373999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a755dd1-a5d1-45e7-a248-5f10c69e7e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.373999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a755dd1-a5d1-45e7-a248-5f10c69e7e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.374492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a755dd1-a5d1-45e7-a248-5f10c69e7e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.375755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a755dd1-a5d1-45e7-a248-5f10c69e7e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.381710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb39058-31ec-467a-81bd-3d270aff74ab None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.382243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb39058-31ec-467a-81bd-3d270aff74ab None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.382715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb39058-31ec-467a-81bd-3d270aff74ab None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.385297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1002/2005] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:14:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:14:34.385904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a755dd1-a5d1-45e7-a248-5f10c69e7e1e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.403549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1004/2006] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:34.405014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb39058-31ec-467a-81bd-3d270aff74ab None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.414719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8271-c384-4b5f-b9ca-0a6a1e32fea8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.415174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8271-c384-4b5f-b9ca-0a6a1e32fea8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.415643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd8271-c384-4b5f-b9ca-0a6a1e32fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.430283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb39058-31ec-467a-81bd-3d270aff74ab None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.435521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.436629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.437012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.437380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.438136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.439857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.440290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-86cd8271-c384-4b5f-b9ca-0a6a1e32fea8 None None] MFA Rules not processed for user `6dad20c291c5440ca6f7c89a3effe871`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:34.441583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-86cd8271-c384-4b5f-b9ca-0a6a1e32fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.458595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1003/2007] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:34.460103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.477206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1ceac-ae81-411b-896c-73d7587993d1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.484467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.485207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.485541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.485822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] PATH_INFO: `/v3/projects/396220daaf3f4a89aab4007f746b96ea/users/8ce2731e0c8647c2a7ea29a0569d2a1d/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.510520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=396220daaf3f4a89aab4007f746b96ea, user_id=8ce2731e0c8647c2a7ea29a0569d2a1d, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.514140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.535422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1004/2008] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:34 2022] PUT /identity/v3/projects/396220daaf3f4a89aab4007f746b96ea/users/8ce2731e0c8647c2a7ea29a0569d2a1d/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:34.539455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.543312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.563447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1005/2009] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:14:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:34.564877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.565204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.565476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.591089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:34.592947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.659453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ed37-4ed9-4071-a1d3-51cff924b8f4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.663907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1006/2010] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:34.665891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.666594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.666878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.667156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] PATH_INFO: `/v3/users/e7c8648de5544757aa1d91c0878c65d2` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.668466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.668871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c8648de5544757aa1d91c0878c65d2)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.670089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.670954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.702839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:34.743510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c8648de5544757aa1d91c0878c65d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:34.758343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c8648de5544757aa1d91c0878c65d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:34.767369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c8648de5544757aa1d91c0878c65d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:34.768203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c8648de5544757aa1d91c0878c65d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:34.768874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c8648de5544757aa1d91c0878c65d2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:34.770301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d6a2d-22db-4f3a-9b63-3dfce561a0f1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.776216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.777415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.777972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.778245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.781292 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.783474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:34.793207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1005/2011] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:34 2022] DELETE /identity/v3/users/e7c8648de5544757aa1d91c0878c65d2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:34.796418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:34.798228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.798600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.799907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.811811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1007/2012] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:34.869496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720a3f8-0458-4884-a877-4519191fe105 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:34.874852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:34.875348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:34.875702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:34.876052 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:34.878153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:34.959882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:34.961411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3715e5-2942-4d1f-8e79-657cfb7d68b2 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:34.972367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1006/2013] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:35.031644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.034990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:35.060581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b9995-8724-4a67-8a83-7840f106120e service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.070847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.072029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.072625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.073247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] PATH_INFO: `/v3/projects/25df8851ae9b4039a00a9a30366ac922` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.075708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25df8851ae9b4039a00a9a30366ac922)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.080423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.181825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25df8851ae9b4039a00a9a30366ac922', 'request_id': 'req-6020362b-c9a8-4336-944d-d677828606e7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:35.192514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25df8851ae9b4039a00a9a30366ac922', 'request_id': 'req-6020362b-c9a8-4336-944d-d677828606e7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:35.200868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25df8851ae9b4039a00a9a30366ac922', 'request_id': 'req-6020362b-c9a8-4336-944d-d677828606e7'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:35.225756 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] The token cache is being invalidate because project 25df8851ae9b4039a00a9a30366ac922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:35.225951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:35.227667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6020362b-c9a8-4336-944d-d677828606e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:35.230918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1008/2014] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:35 2022] DELETE /identity/v3/projects/25df8851ae9b4039a00a9a30366ac922 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:35.238870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.239418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.239668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.263548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:35.264823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:35.377953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1007/2015] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:35.386043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.386398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.386602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.417713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:35.418941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:35.431341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1009/2016] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:35.432247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.444252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edec11-dd82-46b6-b414-0f6cad66a2d5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.450236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.450500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.450853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.451262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.452270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:35.452699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.454522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.470762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1008/2017] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:14:35.474807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.486010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975c92b-2e3f-42eb-b47c-5eac74e2a847 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.494065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.494714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.495331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.495737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.496544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.499158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.529760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1010/2018] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:35 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:35.534229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.547718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7caaa-cfda-40ef-97a0-fd3f8f80b20e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.587877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:35.649631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.650603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.651058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.651571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.652831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.656862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.695569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1009/2019] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:35 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:35.699707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.715726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b89a33-1510-4905-8a9b-21a213c97a5a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.722395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bae16-c4a4-4e28-8d88-4b103efe324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.723381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bae16-c4a4-4e28-8d88-4b103efe324b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.723818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bae16-c4a4-4e28-8d88-4b103efe324b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.724269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bae16-c4a4-4e28-8d88-4b103efe324b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.725171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bae16-c4a4-4e28-8d88-4b103efe324b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.727193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bae16-c4a4-4e28-8d88-4b103efe324b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.741042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1011/2020] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:35.746076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.766066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a63b45-02dc-490b-94ab-82b63bdf1413 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.776313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.777476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.777847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.778390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.779478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.781683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.795752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1010/2021] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:35.797403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.798095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.798699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.800027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.830411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4823f-dde1-4cd0-ba75-4f205bd6d1a2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:35.835816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165eaa5-f7f3-4272-a274-236b5d227aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:35.836535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165eaa5-f7f3-4272-a274-236b5d227aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.836816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165eaa5-f7f3-4272-a274-236b5d227aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.837130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165eaa5-f7f3-4272-a274-236b5d227aac admin admin] PATH_INFO: `/v3/projects/0ec91c82336b4c4e8feaed854661a241/users/b7772807d1d54f5fac8e0331ed485e93/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.856678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165eaa5-f7f3-4272-a274-236b5d227aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec91c82336b4c4e8feaed854661a241, user_id=b7772807d1d54f5fac8e0331ed485e93, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:35.860811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165eaa5-f7f3-4272-a274-236b5d227aac admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:35.863143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] MFA Rules not processed for user `8ce2731e0c8647c2a7ea29a0569d2a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:35.864502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:35.877923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1011/2022] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:35 2022] PUT /identity/v3/projects/0ec91c82336b4c4e8feaed854661a241/users/b7772807d1d54f5fac8e0331ed485e93/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:35.897194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:35.897559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:35.897844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:35.932128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:35.939099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] MFA Rules not processed for user `b7772807d1d54f5fac8e0331ed485e93`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:35.941433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:35.975931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1012/2023] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:35.989113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:35.990571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:36.037106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1012/2024] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:36.047694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:36.049124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:36.056654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb05b31-0bd0-4228-bb35-f86e9110b550 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:36.107897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:36.108825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:36.109146 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:36.109417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:36.110808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369c8a3-6481-4d5d-bbe3-c19a1168ff09 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:36.114541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a835781-142d-4996-82c9-ad52ab0b4307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:36.115177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a835781-142d-4996-82c9-ad52ab0b4307 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:36.115465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a835781-142d-4996-82c9-ad52ab0b4307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:36.115743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a835781-142d-4996-82c9-ad52ab0b4307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:36.125521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a835781-142d-4996-82c9-ad52ab0b4307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:36.126984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a835781-142d-4996-82c9-ad52ab0b4307 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:36.136316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1013/2025] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:36.171041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:36.171539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:36.182565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1013/2026] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:38.697452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a7fd3-0b31-4042-b39e-cb3de2136c46 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:38.697978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a7fd3-0b31-4042-b39e-cb3de2136c46 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:38.697978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a7fd3-0b31-4042-b39e-cb3de2136c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:38.722480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-546a7fd3-0b31-4042-b39e-cb3de2136c46 None None] MFA Rules not processed for user `8ce2731e0c8647c2a7ea29a0569d2a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:38.723712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-546a7fd3-0b31-4042-b39e-cb3de2136c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:38.730934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1014/2027] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:39.946049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:39.947516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:40.029813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:40.048167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8868f-be09-4036-83d1-a707044653e7 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:40.053125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:40.053803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:40.054081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:40.054353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] PATH_INFO: `/v3/users/6373c81be86e4be58cf669d2f054d2ee` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:40.055662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6373c81be86e4be58cf669d2f054d2ee)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:40.057723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:40.124238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6373c81be86e4be58cf669d2f054d2ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.124949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6373c81be86e4be58cf669d2f054d2ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.139640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6373c81be86e4be58cf669d2f054d2ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.149818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6373c81be86e4be58cf669d2f054d2ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.162772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6373c81be86e4be58cf669d2f054d2ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.171712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1014/2028] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:39 2022] DELETE /identity/v3/users/6373c81be86e4be58cf669d2f054d2ee => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:40.176912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:40.177905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:40.178186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:40.199863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:40.201689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:40.214608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:40.278656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1015/2029] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:40.289469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:40.290903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:40.361980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f2be4-7a37-4e44-b0b1-c4153f245058 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:40.365664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:40.366382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:40.366700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:40.366995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:40.376930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:40.378414 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:40.388990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1015/2030] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:40.843154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:40.845208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:40.908257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656117b-6edb-4a7f-ab77-a779c5a35397 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:40.914530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:40.915211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:40.915498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:40.915776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] PATH_INFO: `/v3/projects/7ed0b97320c94c55910a804ee0fcb2ff` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:40.917091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed0b97320c94c55910a804ee0fcb2ff)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:40.919062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:40.975577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed0b97320c94c55910a804ee0fcb2ff', 'request_id': 'req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.983113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed0b97320c94c55910a804ee0fcb2ff', 'request_id': 'req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:40.990861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed0b97320c94c55910a804ee0fcb2ff', 'request_id': 'req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.021289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] The token cache is being invalidate because project 7ed0b97320c94c55910a804ee0fcb2ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:41.021853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.022883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.024660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1016/2031] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:40 2022] DELETE /identity/v3/projects/7ed0b97320c94c55910a804ee0fcb2ff => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:41.029618 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:41.040367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1df80-0675-4474-8885-ce714a14a47c service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:41.147551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:41.148801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:41.149666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:41.150457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] PATH_INFO: `/v3/users/87957bb7be294726a25c39c052049449` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:41.152775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87957bb7be294726a25c39c052049449)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:41.157483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:41.192702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:41.268793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87957bb7be294726a25c39c052049449'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.270552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87957bb7be294726a25c39c052049449'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.299682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87957bb7be294726a25c39c052049449'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.313094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87957bb7be294726a25c39c052049449'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.329199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-72af7d8e-553c-4df0-bf0a-b919434d92f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87957bb7be294726a25c39c052049449'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:41.343884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1016/2032] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:41 2022] DELETE /identity/v3/users/87957bb7be294726a25c39c052049449 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:41.906003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:41.930913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:41.971734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c862d-cb2b-40ae-9f0b-a04ef9dff24a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.087649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.089706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.090173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.090856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] PATH_INFO: `/v3/projects/483457a33a384c2c844ec6de1f141593` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.093734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483457a33a384c2c844ec6de1f141593)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.096359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.163654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:42.177405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483457a33a384c2c844ec6de1f141593', 'request_id': 'req-f17bb4f0-93ca-4d96-80d0-d4771926ed19'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:42.197941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483457a33a384c2c844ec6de1f141593', 'request_id': 'req-f17bb4f0-93ca-4d96-80d0-d4771926ed19'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:42.206456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483457a33a384c2c844ec6de1f141593', 'request_id': 'req-f17bb4f0-93ca-4d96-80d0-d4771926ed19'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:42.228214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] The token cache is being invalidate because project 483457a33a384c2c844ec6de1f141593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:42.228695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:42.229551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f17bb4f0-93ca-4d96-80d0-d4771926ed19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:42.231451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1017/2033] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:41 2022] DELETE /identity/v3/projects/483457a33a384c2c844ec6de1f141593 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:42.237109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.237697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.237790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.260796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:42.262105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:42.356430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1017/2034] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:42.367939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.368208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.368395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.391889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:42.393165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:42.404275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1018/2035] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:42.411887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:42.431620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c108d-49db-400f-93f9-3f139835fd71 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.440222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.441444 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.441926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.442405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.443697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:42.444221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.445934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.465640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1018/2036] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:42.467384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:42.476535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee47b69-7020-4ed6-814c-25772e705e5c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.483462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.484099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.484379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.484651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.485381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.487403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.530812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1019/2037] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:42 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:42.536849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:42.548767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a95a-4251-4126-948b-c1a8baa52be7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.645069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.645845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.646108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.646545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.647325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.649726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.674597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1019/2038] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:42 2022] POST /identity/v3/users => generated 283 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:42.679432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:42.702093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c1d5c-0cb3-40e0-a5c0-2f0bc23a5617 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.712558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.715556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.716264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.716767 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.719272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.722374 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.744178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1020/2039] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:42.749877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:42.762299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469054b4-06bb-44db-aef1-bb03286e21c3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.773289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12197142-8f5d-4942-9620-ce864f5694f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.773935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12197142-8f5d-4942-9620-ce864f5694f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.774220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12197142-8f5d-4942-9620-ce864f5694f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.774538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12197142-8f5d-4942-9620-ce864f5694f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.775322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12197142-8f5d-4942-9620-ce864f5694f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.777664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12197142-8f5d-4942-9620-ce864f5694f1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.789437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:42.807744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:42.812925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1020/2040] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:42.822626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3392761-5f00-4065-8450-6ce95fc4ff93 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:42.836249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:42.837153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.837444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.837809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] PATH_INFO: `/v3/projects/14f8f45763644abd94687c47b41ea184/users/aa32990e8f5440418f8958b12c4f3268/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.861046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f8f45763644abd94687c47b41ea184, user_id=aa32990e8f5440418f8958b12c4f3268, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:42.863999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:42.896990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1021/2041] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:42 2022] PUT /identity/v3/projects/14f8f45763644abd94687c47b41ea184/users/aa32990e8f5440418f8958b12c4f3268/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:42.902691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:42.902973 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:42.903279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:42.945349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] MFA Rules not processed for user `aa32990e8f5440418f8958b12c4f3268`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:42.946652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:43.058147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1021/2042] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:43.059381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.156280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f0f70-b219-4bf5-90a8-faa2a66ed50a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.167442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.168219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.168500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.168745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.169854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.173297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.227966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1022/2043] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:43 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:43.233044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.248438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3045d85-de91-4e46-a9f0-27bdf34e4fd3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.329429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.330500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.331015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.331544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.332902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.336752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.381506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1022/2044] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:43 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:43.385920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.398253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df043f3c-5f4b-4a38-946a-999f21f4d247 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.410096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.411110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.411548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.411982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.412883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.414578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.431623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1023/2045] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:43.436256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.452906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0805d-9e25-462c-8ff7-5422991e73c2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.461797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.463081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.463585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.464098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] PATH_INFO: `/v3/projects/91b87daac2d34f898224b46a527da5d9/users/eaef9ff670a0495f8bd9046d40dd9d00/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.503798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b87daac2d34f898224b46a527da5d9, user_id=eaef9ff670a0495f8bd9046d40dd9d00, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.506790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.528627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1023/2046] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:43 2022] PUT /identity/v3/projects/91b87daac2d34f898224b46a527da5d9/users/eaef9ff670a0495f8bd9046d40dd9d00/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:43.533939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.565613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8bffc-84b9-4844-bad1-a62319c1876c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.609127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:43.640637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fb2d7-2f2f-4f06-b564-82190838697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.641325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb2d7-2f2f-4f06-b564-82190838697b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.641609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb2d7-2f2f-4f06-b564-82190838697b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.641886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb2d7-2f2f-4f06-b564-82190838697b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.642625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fb2d7-2f2f-4f06-b564-82190838697b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.644345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fb2d7-2f2f-4f06-b564-82190838697b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.655844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1024/2047] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:43.660155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.671823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579e57-41b3-40fd-b415-c2a56c0ec6c9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.677020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.677897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.678184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.678458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] PATH_INFO: `/v3/projects/91b87daac2d34f898224b46a527da5d9/users/eaef9ff670a0495f8bd9046d40dd9d00/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.680476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91b87daac2d34f898224b46a527da5d9, user_id=eaef9ff670a0495f8bd9046d40dd9d00, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.683211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.698005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1024/2048] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:43 2022] PUT /identity/v3/projects/91b87daac2d34f898224b46a527da5d9/users/eaef9ff670a0495f8bd9046d40dd9d00/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:43.710360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b623b-484e-476f-b67b-3d584f93745d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.710676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b623b-484e-476f-b67b-3d584f93745d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.711059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b623b-484e-476f-b67b-3d584f93745d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.743759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-254b623b-484e-476f-b67b-3d584f93745d None None] MFA Rules not processed for user `eaef9ff670a0495f8bd9046d40dd9d00`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:43.745523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-254b623b-484e-476f-b67b-3d584f93745d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:43.858188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1025/2049] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:43.874590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:43.876697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:43.937903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:43.954416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201e3a3-7149-4d31-9761-6af0ac532bf2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:43.959164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b0e27-d7ca-40a3-962a-55a741060cf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:43.959871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b0e27-d7ca-40a3-962a-55a741060cf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:43.960235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b0e27-d7ca-40a3-962a-55a741060cf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:43.960515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b0e27-d7ca-40a3-962a-55a741060cf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:43.973912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b0e27-d7ca-40a3-962a-55a741060cf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:43.975456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b0e27-d7ca-40a3-962a-55a741060cf6 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:43.990078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1025/2050] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:43 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:44.470887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b623b-484e-476f-b67b-3d584f93745d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:44.541515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b623b-484e-476f-b67b-3d584f93745d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:44.547461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:44.548146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:44.548422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:44.548698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] PATH_INFO: `/v3/users/aa32990e8f5440418f8958b12c4f3268` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:44.549901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa32990e8f5440418f8958b12c4f3268)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:44.551928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:44.632756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa32990e8f5440418f8958b12c4f3268'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:44.650400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa32990e8f5440418f8958b12c4f3268'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:44.659608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:44.668383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa32990e8f5440418f8958b12c4f3268'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:44.670124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa32990e8f5440418f8958b12c4f3268'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:44.671716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa32990e8f5440418f8958b12c4f3268'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:44.708217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1026/2051] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:44 2022] DELETE /identity/v3/users/aa32990e8f5440418f8958b12c4f3268 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:44.713656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:44.714247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:44.714247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:44.735419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:44.737154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:44.814593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1026/2052] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:44.827952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:44.829528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:44.899804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b45d9-2cea-40df-a54b-9ab21917e4aa admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:44.904420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:44.905151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:44.905363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:44.905633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:44.915784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:44.917456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:44.929287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1027/2053] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:45.312933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:45.314757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:45.337535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bdcc3-f27d-4f90-9c47-b1cd2b9e5799 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:45.378679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:45.379751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:45.380307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:45.380703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] PATH_INFO: `/v3/projects/14f8f45763644abd94687c47b41ea184` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:45.384207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f8f45763644abd94687c47b41ea184)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:45.393290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:45.461633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f8f45763644abd94687c47b41ea184', 'request_id': 'req-4ed22009-938c-448c-8232-5990e69075c0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.471922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f8f45763644abd94687c47b41ea184', 'request_id': 'req-4ed22009-938c-448c-8232-5990e69075c0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.479390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f8f45763644abd94687c47b41ea184', 'request_id': 'req-4ed22009-938c-448c-8232-5990e69075c0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.492088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] The token cache is being invalidate because project 14f8f45763644abd94687c47b41ea184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:45.492380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.493081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.494642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1027/2054] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:45 2022] DELETE /identity/v3/projects/14f8f45763644abd94687c47b41ea184 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:45.500138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:45.513127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14b27b-371d-44dc-bda2-edbc303a35d0 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:45.613701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:45.614367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:45.614650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:45.614954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] PATH_INFO: `/v3/users/eaef9ff670a0495f8bd9046d40dd9d00` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:45.616198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaef9ff670a0495f8bd9046d40dd9d00)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:45.618239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:45.689118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaef9ff670a0495f8bd9046d40dd9d00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.703657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaef9ff670a0495f8bd9046d40dd9d00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.716199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaef9ff670a0495f8bd9046d40dd9d00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.717451 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaef9ff670a0495f8bd9046d40dd9d00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.718223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaef9ff670a0495f8bd9046d40dd9d00'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:45.735694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:45.740746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1028/2055] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:45 2022] DELETE /identity/v3/users/eaef9ff670a0495f8bd9046d40dd9d00 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:45.804565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:45.806738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:45.830295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed22009-938c-448c-8232-5990e69075c0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:45.941165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:45.942276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:45.942797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:45.943331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] PATH_INFO: `/v3/users/b7772807d1d54f5fac8e0331ed485e93` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:45.947108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7772807d1d54f5fac8e0331ed485e93)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:45.950581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:45.975539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:46.040542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7772807d1d54f5fac8e0331ed485e93'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.041217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7772807d1d54f5fac8e0331ed485e93'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.056882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7772807d1d54f5fac8e0331ed485e93'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.066360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7772807d1d54f5fac8e0331ed485e93'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.075114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-507e82e2-b509-4499-9668-13315ccf6c88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7772807d1d54f5fac8e0331ed485e93'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.077622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.079359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.085825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1028/2056] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:45 2022] DELETE /identity/v3/users/b7772807d1d54f5fac8e0331ed485e93 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:46.087370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.087699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.088323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.098193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618c50-dcdf-4015-b60a-9c5ea55f00cb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.116306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:46.117603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.195719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.196480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.196771 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.197080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] PATH_INFO: `/v3/projects/91b87daac2d34f898224b46a527da5d9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.198388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91b87daac2d34f898224b46a527da5d9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.200532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.208978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1029/2057] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:46.222915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.226046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.272790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91b87daac2d34f898224b46a527da5d9', 'request_id': 'req-887bb3df-03b0-4fd5-9a88-8e70bba0f358'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.281601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91b87daac2d34f898224b46a527da5d9', 'request_id': 'req-887bb3df-03b0-4fd5-9a88-8e70bba0f358'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.288798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91b87daac2d34f898224b46a527da5d9', 'request_id': 'req-887bb3df-03b0-4fd5-9a88-8e70bba0f358'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.304727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5cce92-791f-481e-9060-77ef0cb9246c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.307192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] The token cache is being invalidate because project 91b87daac2d34f898224b46a527da5d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:46.307628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.308350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-887bb3df-03b0-4fd5-9a88-8e70bba0f358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:46.310976 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1029/2058] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:46 2022] DELETE /identity/v3/projects/91b87daac2d34f898224b46a527da5d9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:46.316194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7092b-41ee-4875-ab3e-abbc487a93b9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.316194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7092b-41ee-4875-ab3e-abbc487a93b9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.316442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7092b-41ee-4875-ab3e-abbc487a93b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.340399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e8f7092b-41ee-4875-ab3e-abbc487a93b9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:46.341838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e8f7092b-41ee-4875-ab3e-abbc487a93b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.366266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:46.380659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.381436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.381784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.382347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.384455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.455107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1030/2059] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:46.462876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.463830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.464673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.476189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.477673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.489535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1030/2060] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:46.491660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:46.492902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.507859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1031/2061] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:46.512775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.523667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86036-ebae-47ba-8878-9ececc755448 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.535635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.536490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.536918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.537346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.538440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:46.538557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.540224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.566941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1031/2062] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:46.573411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.590464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278fea7-72ad-4b1f-b3d8-f364551d156c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.601891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.602989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.603435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.603781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.605071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.609232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.656500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1032/2063] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:46 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:46.658953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.674155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d96288-103f-489a-ae82-4b2786a7307e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.756241 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.756824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.757153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.757494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.758159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.760271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.795281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1032/2064] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:46 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:46.796809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.803389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:46.814432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ff73c-948b-474f-9e6f-d9ac2c0a6953 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.823989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.824834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.825786 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.826794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.828262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.830043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.851731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1033/2065] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:46.857366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.871966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb20cc-0f03-46db-8119-06911768db90 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.885518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.886284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.886917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.887507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.890455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.894663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:46.908664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1033/2066] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:46.920056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.944329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:46.946735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:46.947049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e250d-4f93-4b9b-a7e8-ed7cd3f7c029 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.956422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa51cc1-39a8-4624-89a1-e341d34275a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.959898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51cc1-39a8-4624-89a1-e341d34275a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.961998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51cc1-39a8-4624-89a1-e341d34275a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.962961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa51cc1-39a8-4624-89a1-e341d34275a9 admin admin] PATH_INFO: `/v3/projects/d60e93cea10340dfb1a446495fe51711/users/00da206844184d00a2d37519f83d1bb8/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.975018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be5075-0b3a-4f0f-b411-6f25c318bb7a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:46.981227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:46.982926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:46.983330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:46.984517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] PATH_INFO: `/v3/projects/0ec91c82336b4c4e8feaed854661a241` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:46.987408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec91c82336b4c4e8feaed854661a241)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:46.990861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.006164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa51cc1-39a8-4624-89a1-e341d34275a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60e93cea10340dfb1a446495fe51711, user_id=00da206844184d00a2d37519f83d1bb8, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.022618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa51cc1-39a8-4624-89a1-e341d34275a9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.062425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1034/2067] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:46 2022] PUT /identity/v3/projects/d60e93cea10340dfb1a446495fe51711/users/00da206844184d00a2d37519f83d1bb8/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:47.078621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.080328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.081029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.089759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec91c82336b4c4e8feaed854661a241', 'request_id': 'req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:47.096458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:47.104692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec91c82336b4c4e8feaed854661a241', 'request_id': 'req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:47.126598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec91c82336b4c4e8feaed854661a241', 'request_id': 'req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:47.139356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] MFA Rules not processed for user `00da206844184d00a2d37519f83d1bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:47.140800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:47.152833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] The token cache is being invalidate because project 0ec91c82336b4c4e8feaed854661a241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:47.153555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:47.155160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-aa730b7c-f9e7-4cbb-a8e3-d56c511e325f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:47.163915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1034/2068] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:46 2022] DELETE /identity/v3/projects/0ec91c82336b4c4e8feaed854661a241 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:47.167506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054b929-f13e-4431-a2b6-085cbc13b420 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.168006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054b929-f13e-4431-a2b6-085cbc13b420 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.168491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054b929-f13e-4431-a2b6-085cbc13b420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.202198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6054b929-f13e-4431-a2b6-085cbc13b420 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:47.205117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6054b929-f13e-4431-a2b6-085cbc13b420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:47.255240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1035/2069] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:47.260613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.266290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:47.288678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1035/2070] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:47.298922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.299259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.299481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.335510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:47.336762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:47.345993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1036/2071] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:47.350644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.363835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc5f12-1ed2-4d34-bb17-9936b5fd3bdd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.369339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.370634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.370918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.371191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.371954 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:47.372227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.374082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.376468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f251f-7cc9-4556-87d6-18006eda6afa None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.382952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.384076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.384195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.384531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.385833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.387387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1037/2072] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:14:47.390640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.392322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.410087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8e10-1cd0-4fb6-9e56-85573dbf5ce5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.415190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.415998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.416147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.416521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.417252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.419227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.441775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1036/2073] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:47.443022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.462243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1038/2074] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/projects => generated 351 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:47.467379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.519029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:47.543945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5212d9-4abf-4f7e-a655-1d5759983f2b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.544774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f4317-d50e-4a6b-b5fd-4042eb42f66b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.549477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.550121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.550404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.550688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.551556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.551911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.552996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.553522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.553636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.554015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.555298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.558674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.580306 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1037/2075] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:47.585236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.590041 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1039/2076] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/users => generated 276 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:47.595693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.599068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c2271-f60e-4808-8de6-e025493aa717 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.605098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.606208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.606539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.606969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.609016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.611201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.614638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463b268-92da-416f-bde5-287730dc33c6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.622534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.623687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.624179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.624680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.626026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.629380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.634884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1038/2077] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:47.636288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.648345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751057a-ed8e-4b0e-b42b-19e8091535f1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.653421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.654852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.655301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.655593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] PATH_INFO: `/v3/projects/996b14d7de8c4116806a80ccf997be9c/users/854196e95a954e62aeb9d34e909d03bd/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.656592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1040/2078] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:47.658330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.674206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] RBAC: Authorizing `identity:create_grant(project_id=996b14d7de8c4116806a80ccf997be9c, user_id=854196e95a954e62aeb9d34e909d03bd, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.677271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.677906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60c464-511a-436b-9f4f-9cbc8c014629 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.687447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.688650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.689588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.690362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.691894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.695052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.696006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1039/2079] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:47 2022] PUT /identity/v3/projects/996b14d7de8c4116806a80ccf997be9c/users/854196e95a954e62aeb9d34e909d03bd/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:47.700704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.718405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1041/2080] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:47.725215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.743765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7939e2e-bd65-42d8-b46f-a0b867ac90fe admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.759569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:47.805529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7371e-4e04-478e-918a-1c9c847622db admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.811217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569918-2390-4513-a4d3-9009882939de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.812432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569918-2390-4513-a4d3-9009882939de admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.812720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569918-2390-4513-a4d3-9009882939de admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.813349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569918-2390-4513-a4d3-9009882939de admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.814057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96569918-2390-4513-a4d3-9009882939de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.817444 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96569918-2390-4513-a4d3-9009882939de admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.826681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b89c4-acc3-44d2-a882-2c7486ac135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.828351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b89c4-acc3-44d2-a882-2c7486ac135d admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.828351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b89c4-acc3-44d2-a882-2c7486ac135d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.828835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b89c4-acc3-44d2-a882-2c7486ac135d admin admin] PATH_INFO: `/v3/projects/b6b61376c7294b7f8ce3a9c269508ec5/users/d3cedd1201e54b35a0febaf5cc79cbaa/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.833363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1040/2081] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:47.834802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569918-2390-4513-a4d3-9009882939de admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:47.848175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569918-2390-4513-a4d3-9009882939de admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:47.853487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f5dca-16b6-4450-9874-152facddb7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:47.854597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f5dca-16b6-4450-9874-152facddb7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.854597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f5dca-16b6-4450-9874-152facddb7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.854949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f5dca-16b6-4450-9874-152facddb7e9 admin admin] PATH_INFO: `/v3/projects/996b14d7de8c4116806a80ccf997be9c/users/854196e95a954e62aeb9d34e909d03bd/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.857363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f5dca-16b6-4450-9874-152facddb7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996b14d7de8c4116806a80ccf997be9c, user_id=854196e95a954e62aeb9d34e909d03bd, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.861073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f5dca-16b6-4450-9874-152facddb7e9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.861317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b89c4-acc3-44d2-a882-2c7486ac135d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b61376c7294b7f8ce3a9c269508ec5, user_id=d3cedd1201e54b35a0febaf5cc79cbaa, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:47.865831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b89c4-acc3-44d2-a882-2c7486ac135d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:47.883750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1041/2082] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:47 2022] PUT /identity/v3/projects/996b14d7de8c4116806a80ccf997be9c/users/854196e95a954e62aeb9d34e909d03bd/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:47.889430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1042/2083] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:47 2022] PUT /identity/v3/projects/b6b61376c7294b7f8ce3a9c269508ec5/users/d3cedd1201e54b35a0febaf5cc79cbaa/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:47.895112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.895402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.895678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.901926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:47.902396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:47.902706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:47.928057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] MFA Rules not processed for user `854196e95a954e62aeb9d34e909d03bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:47.928699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:47.935794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] MFA Rules not processed for user `d3cedd1201e54b35a0febaf5cc79cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:47.937107 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:48.008460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1042/2084] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:48.020677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.022239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:48.026803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1043/2085] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3218 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:48.028065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.031426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:48.108253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d712cb-f0e6-44af-9cf0-5f13529ebf7c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.112428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.113092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.113373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.113646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.124646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83174cdc-4062-412c-a3c6-aa678000c8aa None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.129961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.132664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.133099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.134102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.134575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.135073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.136269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.140597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.150676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1043/2086] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:48.188037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1044/2087] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:48 2022] POST /identity/v3/projects => generated 357 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:48.194187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.226003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e764ad2-f45b-4e71-960a-8abc3e4b6b2e service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.247144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.250384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:48.260566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:48.325216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.325464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d3c97-c0c8-4f5b-9d84-d170d83535c8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.327750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.328389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.329055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.331084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.331386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.331881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.332235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.332578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.354998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.390521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1044/2088] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:48 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:48.394695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.407534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda7d28-335c-4dce-a627-eb754c11ee05 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.416909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.418002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.418371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.420236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.422496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.426289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.436470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.438014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.447945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1045/2089] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:48.449256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1045/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:48.452743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.463829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e76b9-bda1-4a70-b337-b67a09a26e17 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.470602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.471292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.471576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.471855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] PATH_INFO: `/v3/projects/5cdc5dd317254b38801d0dfb4930617b/users/6d199b85be7542b2ad3c5f03f82efddf/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.493385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdc5dd317254b38801d0dfb4930617b, user_id=6d199b85be7542b2ad3c5f03f82efddf, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.496382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.515803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1046/2091] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:48 2022] PUT /identity/v3/projects/5cdc5dd317254b38801d0dfb4930617b/users/6d199b85be7542b2ad3c5f03f82efddf/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:48.519605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.532488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654ecf3-6156-4784-86c1-fadb065933ef service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.613347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654348fc-f6f4-493e-a4cd-9c0eadbe0a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.614077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654348fc-f6f4-493e-a4cd-9c0eadbe0a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.614434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654348fc-f6f4-493e-a4cd-9c0eadbe0a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.614892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654348fc-f6f4-493e-a4cd-9c0eadbe0a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.615593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654348fc-f6f4-493e-a4cd-9c0eadbe0a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.617474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654348fc-f6f4-493e-a4cd-9c0eadbe0a97 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.636660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1046/2092] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:48.644647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:48.653492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e69bd0-2528-4b26-8bfe-be06373b14b5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:48.659408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:48.660237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.660989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.661130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] PATH_INFO: `/v3/projects/5cdc5dd317254b38801d0dfb4930617b/users/6d199b85be7542b2ad3c5f03f82efddf/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.663552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdc5dd317254b38801d0dfb4930617b, user_id=6d199b85be7542b2ad3c5f03f82efddf, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:48.673672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:48.693039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1047/2093] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:48 2022] PUT /identity/v3/projects/5cdc5dd317254b38801d0dfb4930617b/users/6d199b85be7542b2ad3c5f03f82efddf/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:48.713470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:48.713953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:48.714432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:48.766389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] MFA Rules not processed for user `6d199b85be7542b2ad3c5f03f82efddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:48.768547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:48.873759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:48.906242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1047/2094] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:48.924975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:49.002437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:49.015025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ace287-ae9b-4c21-8c26-24f83fcfd440 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:49.023302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:49.024403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:49.024907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:49.025445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:49.046934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:49.050174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:49.067794 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1048/2095] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:48 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:49.181033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:49.191990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3646377-1ec9-4baf-a901-48f75b8c6e1b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:49.196074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30038edf-7e50-4945-b49f-d756dcc503c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:49.196739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038edf-7e50-4945-b49f-d756dcc503c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:49.197165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038edf-7e50-4945-b49f-d756dcc503c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:49.197455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30038edf-7e50-4945-b49f-d756dcc503c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:49.262335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30038edf-7e50-4945-b49f-d756dcc503c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:49.263938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30038edf-7e50-4945-b49f-d756dcc503c1 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:49.274758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1048/2096] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:49 2022] GET /identity/v3/auth/tokens => generated 3218 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:52.769156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:52.859277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb8f36-4ed1-4f8e-881d-5bbef3c823ae service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:52.864229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:52.864903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:52.865222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:52.865499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] PATH_INFO: `/v3/users/00da206844184d00a2d37519f83d1bb8` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:52.866736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00da206844184d00a2d37519f83d1bb8)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:52.868722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:52.931389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00da206844184d00a2d37519f83d1bb8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:52.941327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00da206844184d00a2d37519f83d1bb8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:52.957251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00da206844184d00a2d37519f83d1bb8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:52.958062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00da206844184d00a2d37519f83d1bb8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:52.958851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00da206844184d00a2d37519f83d1bb8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:52.982626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1049/2097] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:52 2022] DELETE /identity/v3/users/00da206844184d00a2d37519f83d1bb8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:52.989074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:52.989721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:52.990066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:53.016625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:53.018080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:53.089678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1049/2098] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:53.100849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:53.102488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:53.171413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145a18-33e5-404a-9309-9bcc2d3b66d1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:53.179310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:53.180292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:53.180744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:53.181239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:53.192467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:53.194965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:53.206921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1050/2099] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:14:53.636692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:53.638553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:53.660095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3baa-dce9-4a10-be84-d648f59fe378 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:53.665729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:53.666508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:53.666860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:53.667203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] PATH_INFO: `/v3/projects/d60e93cea10340dfb1a446495fe51711` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:53.668548 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60e93cea10340dfb1a446495fe51711)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:53.671269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:53.751796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60e93cea10340dfb1a446495fe51711', 'request_id': 'req-ace843bb-f004-43dc-92e5-f81ac7e22aa1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.761506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60e93cea10340dfb1a446495fe51711', 'request_id': 'req-ace843bb-f004-43dc-92e5-f81ac7e22aa1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.769667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60e93cea10340dfb1a446495fe51711', 'request_id': 'req-ace843bb-f004-43dc-92e5-f81ac7e22aa1'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.782335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] The token cache is being invalidate because project d60e93cea10340dfb1a446495fe51711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:53.782742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.783293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.784947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1050/2100] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:53 2022] DELETE /identity/v3/projects/d60e93cea10340dfb1a446495fe51711 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:53.789739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:53.804447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9a9dd-403b-4415-985b-8cf37b74756d service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:53.824302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:53.896707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:53.897504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:53.897964 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:53.898383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] PATH_INFO: `/v3/users/854196e95a954e62aeb9d34e909d03bd` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:53.899718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854196e95a954e62aeb9d34e909d03bd)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:53.901997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:53.978180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854196e95a954e62aeb9d34e909d03bd'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.988737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854196e95a954e62aeb9d34e909d03bd'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:53.999827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854196e95a954e62aeb9d34e909d03bd'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.001159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854196e95a954e62aeb9d34e909d03bd'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.002229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5056b8c4-b591-4b32-b948-eebc952cd346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854196e95a954e62aeb9d34e909d03bd'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.039650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1051/2101] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:53 2022] DELETE /identity/v3/users/854196e95a954e62aeb9d34e909d03bd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:54.531303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:54.534902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:54.551680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:54.563416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace843bb-f004-43dc-92e5-f81ac7e22aa1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:54.668563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:54.670326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:54.671450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:54.671879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] PATH_INFO: `/v3/projects/996b14d7de8c4116806a80ccf997be9c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:54.673663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996b14d7de8c4116806a80ccf997be9c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:54.688937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:54.749218 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996b14d7de8c4116806a80ccf997be9c', 'request_id': 'req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.759114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996b14d7de8c4116806a80ccf997be9c', 'request_id': 'req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.767264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996b14d7de8c4116806a80ccf997be9c', 'request_id': 'req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.779870 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] The token cache is being invalidate because project 996b14d7de8c4116806a80ccf997be9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:54.780179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.781047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d70af4d-b0ea-4f74-98af-8f425a2c7eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:54.782639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1051/2102] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:54 2022] DELETE /identity/v3/projects/996b14d7de8c4116806a80ccf997be9c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:54.789908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:54.790755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:54.791244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:54.814498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:54.815879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:54.902872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1052/2103] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:54.912696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:54.913207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:54.913681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:54.947411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:54.949519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:54.961103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1052/2104] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:54.965564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:54.978840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712bc38-1a49-43b0-a472-4446ab794095 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:54.984082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:54.984719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:54.985221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:54.985492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:54.986239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:54.986660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:54.988160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.003565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1053/2105] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:14:55.009397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.021118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeba8c-bc17-4d41-b360-1bed5d738819 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.026884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.027970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.028522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.029149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.030206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.032894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.073699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1053/2106] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:55 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:55.078631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.092027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c8f5d-87b5-4e3c-a2fd-b7ed61c3d879 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.116520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:55.181795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.182721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.183122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.183471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.184415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.187998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.228775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1054/2107] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:55 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:55.230998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.247163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92b73c-f74c-4cda-844b-a734575dfef1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.255930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.256847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.257304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.257698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.258742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.261288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.286231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1054/2108] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:14:55.291436 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.310040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15068396-1346-4e24-bd89-6817e5999a93 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.319025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb2809-79b1-4bb6-97e9-9bd84e8e42d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.320647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb2809-79b1-4bb6-97e9-9bd84e8e42d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.321083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb2809-79b1-4bb6-97e9-9bd84e8e42d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.321490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb2809-79b1-4bb6-97e9-9bd84e8e42d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.322558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb2809-79b1-4bb6-97e9-9bd84e8e42d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.344775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb2809-79b1-4bb6-97e9-9bd84e8e42d7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.360837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1055/2109] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:55.364887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.395968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624f463-d40b-46b1-9260-c733d0f9f7b6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.416289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.417370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.417625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.417984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] PATH_INFO: `/v3/projects/2c2ebdcaa6844292884dc6529c8fae93/users/8d4d4dfaa07241918c47920fab48d516/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.438995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c2ebdcaa6844292884dc6529c8fae93, user_id=8d4d4dfaa07241918c47920fab48d516, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.445697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.475520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1055/2110] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:55 2022] PUT /identity/v3/projects/2c2ebdcaa6844292884dc6529c8fae93/users/8d4d4dfaa07241918c47920fab48d516/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:55.490321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.490733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.491132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.548201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] MFA Rules not processed for user `8d4d4dfaa07241918c47920fab48d516`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:55.550454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:55.674055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1056/2111] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:55.678585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.723650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:55.782567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ce248-8034-4361-9c12-392b6cb9521a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.799109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.799884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.800182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.800441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.801280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.803339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.845623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1056/2112] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:55 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:55.849540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.862022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610257fe-5da1-459e-b725-c7a044a9ce5d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:55.942194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:55.942728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:55.943007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:55.943289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:55.944061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:55.946091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:55.971714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1057/2113] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:55 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:14:55.977652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:55.991613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70bcfa2-372f-4e45-a3db-cc86895affad admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:56.001216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:56.002101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.002477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.002753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.003779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:56.005895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:56.017680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1057/2114] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:56.022563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:56.034251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dbfc5-2f3c-4261-9629-9838d3368bc7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:56.039262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:56.039957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.040236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.040542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] PATH_INFO: `/v3/projects/cb7363099ebc4f718259e584abc06f1f/users/d7ffebabb40844c895ac2e264da99a6a/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.059790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7363099ebc4f718259e584abc06f1f, user_id=d7ffebabb40844c895ac2e264da99a6a, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:56.063000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:56.065626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:56.079897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1058/2115] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:56 2022] PUT /identity/v3/projects/cb7363099ebc4f718259e584abc06f1f/users/d7ffebabb40844c895ac2e264da99a6a/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:14:56.084493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:56.096789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438016bb-a3f3-4c3c-90e5-7d0e8d9e8e1b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:56.167821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc615907-4c19-4da3-a332-a63cf5de5291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:56.168459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc615907-4c19-4da3-a332-a63cf5de5291 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.168738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc615907-4c19-4da3-a332-a63cf5de5291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.169051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc615907-4c19-4da3-a332-a63cf5de5291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.169815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc615907-4c19-4da3-a332-a63cf5de5291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:56.171841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc615907-4c19-4da3-a332-a63cf5de5291 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:56.186828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1058/2116] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:14:56.188364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:56.202253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b95af5-28bc-4d99-b38a-475d85f40b2c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:56.208421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:56.209142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.209419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.209693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] PATH_INFO: `/v3/projects/cb7363099ebc4f718259e584abc06f1f/users/d7ffebabb40844c895ac2e264da99a6a/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.211993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7363099ebc4f718259e584abc06f1f, user_id=d7ffebabb40844c895ac2e264da99a6a, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:56.216689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:56.230544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1059/2117] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:14:56 2022] PUT /identity/v3/projects/cb7363099ebc4f718259e584abc06f1f/users/d7ffebabb40844c895ac2e264da99a6a/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:56.242340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac79e71-5814-4d75-830b-ddb2c34281cf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.242340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac79e71-5814-4d75-830b-ddb2c34281cf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.242800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac79e71-5814-4d75-830b-ddb2c34281cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.276739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2ac79e71-5814-4d75-830b-ddb2c34281cf None None] MFA Rules not processed for user `d7ffebabb40844c895ac2e264da99a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:56.278182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2ac79e71-5814-4d75-830b-ddb2c34281cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:56.380989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1059/2118] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:56.394596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:56.396547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:56.488411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2b6e4-20ab-44c6-9292-e392b474105c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:56.493621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:56.494278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.494559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.494832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.507889 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:56.509437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:56.527578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1060/2119] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:56.808228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5a44d-4dcf-41a0-a1ad-d11c8dd952dc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:56.808679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5a44d-4dcf-41a0-a1ad-d11c8dd952dc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:56.808774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5a44d-4dcf-41a0-a1ad-d11c8dd952dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:56.831641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-53e5a44d-4dcf-41a0-a1ad-d11c8dd952dc None None] MFA Rules not processed for user `8ce2731e0c8647c2a7ea29a0569d2a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:56.833639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-53e5a44d-4dcf-41a0-a1ad-d11c8dd952dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:56.865914 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:56.949860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1060/2120] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:57.471850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:57.566073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356dc105-d33c-4a9f-8623-5abf0b263562 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:57.572588 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:57.573297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:57.573591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:57.574005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] PATH_INFO: `/v3/users/8d4d4dfaa07241918c47920fab48d516` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:57.575411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4d4dfaa07241918c47920fab48d516)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:57.578033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:57.652029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4d4dfaa07241918c47920fab48d516'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:57.662483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4d4dfaa07241918c47920fab48d516'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:57.672632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4d4dfaa07241918c47920fab48d516'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:57.673490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4d4dfaa07241918c47920fab48d516'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:57.674408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4d4dfaa07241918c47920fab48d516'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:57.700343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1061/2121] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:57 2022] DELETE /identity/v3/users/8d4d4dfaa07241918c47920fab48d516 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:57.705916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:57.706385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:57.706754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:57.738648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:57.739238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:57.817155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1061/2122] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:14:57.826888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:57.828679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:57.839066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:57.921421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a772a-5c5c-45c5-b1a8-922e6318c2c4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:57.925345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:57.925980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:57.926260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:57.926532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:57.937093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:57.938578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:57.951972 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1062/2123] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:14:58.457911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:58.459918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:58.480180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb51f92-2c85-4801-a9f0-0ab30e6314e4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:58.489425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:58.490375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:58.490971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:58.491407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] PATH_INFO: `/v3/projects/2c2ebdcaa6844292884dc6529c8fae93` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:58.492927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c2ebdcaa6844292884dc6529c8fae93)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:58.495515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:58.571977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c2ebdcaa6844292884dc6529c8fae93', 'request_id': 'req-e9ea06fd-b58a-4275-94f8-cca4655f877f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.586402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c2ebdcaa6844292884dc6529c8fae93', 'request_id': 'req-e9ea06fd-b58a-4275-94f8-cca4655f877f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.597542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c2ebdcaa6844292884dc6529c8fae93', 'request_id': 'req-e9ea06fd-b58a-4275-94f8-cca4655f877f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.637341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] The token cache is being invalidate because project 2c2ebdcaa6844292884dc6529c8fae93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:58.637731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.639847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.643000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1062/2124] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:58 2022] DELETE /identity/v3/projects/2c2ebdcaa6844292884dc6529c8fae93 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:58.649220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:58.668998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b362d-c0bf-4688-9b7f-79d19bfb947d service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:58.796213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:58.797304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:58.797958 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:58.798376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] PATH_INFO: `/v3/users/d7ffebabb40844c895ac2e264da99a6a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:58.800336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ffebabb40844c895ac2e264da99a6a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:58.803300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:58.876846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ffebabb40844c895ac2e264da99a6a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.884708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ffebabb40844c895ac2e264da99a6a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.896469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ffebabb40844c895ac2e264da99a6a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.897188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ffebabb40844c895ac2e264da99a6a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.897829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8af1a30c-73ec-4821-81c7-e194a92caddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ffebabb40844c895ac2e264da99a6a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:58.929072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1063/2125] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:14:58 2022] DELETE /identity/v3/users/d7ffebabb40844c895ac2e264da99a6a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:59.273376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:59.276252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:59.312236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea06fd-b58a-4275-94f8-cca4655f877f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:59.383584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:59.394450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:59.395389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:59.395806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:59.396295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] PATH_INFO: `/v3/projects/cb7363099ebc4f718259e584abc06f1f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:59.398447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7363099ebc4f718259e584abc06f1f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:59.403844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:59.476439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7363099ebc4f718259e584abc06f1f', 'request_id': 'req-93f29da8-9a62-4611-bfef-680b5e76d306'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:59.486723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7363099ebc4f718259e584abc06f1f', 'request_id': 'req-93f29da8-9a62-4611-bfef-680b5e76d306'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:59.494673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7363099ebc4f718259e584abc06f1f', 'request_id': 'req-93f29da8-9a62-4611-bfef-680b5e76d306'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:59.508734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] The token cache is being invalidate because project cb7363099ebc4f718259e584abc06f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:14:59.509062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:59.509892 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-93f29da8-9a62-4611-bfef-680b5e76d306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:14:59.511504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1063/2126] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:14:59 2022] DELETE /identity/v3/projects/cb7363099ebc4f718259e584abc06f1f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:14:59.517752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:59.518101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:59.518371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:59.540263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:59.541742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:59.581917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:14:59.651462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1064/2127] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:59.663401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:59.663698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:59.664027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:59.699521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:14:59.704898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:14:59.721917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1064/2128] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:14:59.727342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:59.752477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0222d-db1d-45fd-a9e4-558d181c3b6a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:59.760839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:59.762006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:59.762522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:59.763013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:59.764573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:14:59.765141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:59.768587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:59.795689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1065/2129] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:14:59.797266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:59.815989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce3990e-bbc4-4145-b6c9-84f47a83c917 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:14:59.822496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:14:59.823298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:14:59.823759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:14:59.824213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:14:59.825348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:14:59.829718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:14:59.878785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1065/2130] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:14:59 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:14:59.884605 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:14:59.901394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96ac4b-5a51-47d7-bf4a-b5e8b378a864 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:00.023878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:00.025758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.026824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.027723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.029414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:00.034470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:00.080381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1066/2131] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:14:59 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:00.088186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:00.105359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a2d94-7f5e-46f1-b188-1cfec46911b9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:00.112435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:00.113455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.113923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.114331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.115222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:00.117273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:00.135234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1066/2132] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:00.142156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:00.161737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf469c-39ab-441a-a8f4-91b922498a8c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:00.167905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f6086-81c6-4cde-8e27-6f96e6bfab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:00.168721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6086-81c6-4cde-8e27-6f96e6bfab80 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.186495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6086-81c6-4cde-8e27-6f96e6bfab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.186495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f6086-81c6-4cde-8e27-6f96e6bfab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.187198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f6086-81c6-4cde-8e27-6f96e6bfab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:00.189106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f6086-81c6-4cde-8e27-6f96e6bfab80 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:00.204452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1067/2133] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:00.209782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:00.228084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee7784-65a5-4050-b9c8-dbd6db2d834c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:00.235060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:00.235770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.236040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.236315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] PATH_INFO: `/v3/projects/d01fe046dd914625a335524f05e3db1d/users/ef0656ff3f504556b873fa9b26cce3db/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.259996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01fe046dd914625a335524f05e3db1d, user_id=ef0656ff3f504556b873fa9b26cce3db, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:00.262835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:00.295064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1067/2134] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:00 2022] PUT /identity/v3/projects/d01fe046dd914625a335524f05e3db1d/users/ef0656ff3f504556b873fa9b26cce3db/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:00.315256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.315816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.316993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.376650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] MFA Rules not processed for user `ef0656ff3f504556b873fa9b26cce3db`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:00.381587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:00.558243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1068/2135] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:00.582305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:00.585586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:00.717445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8c319-ff30-40ae-a8b5-90bbf181f7f1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:00.722384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6c629-6acb-4ae5-8601-427ad3390647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:00.722828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6c629-6acb-4ae5-8601-427ad3390647 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.723233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6c629-6acb-4ae5-8601-427ad3390647 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.723499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6c629-6acb-4ae5-8601-427ad3390647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.736589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6c629-6acb-4ae5-8601-427ad3390647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:00.738328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6c629-6acb-4ae5-8601-427ad3390647 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:00.751371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1068/2136] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:00.791091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:00.826363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:00.885943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e3336-e403-42c7-bc09-5faa0d3caa62 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:00.891585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:00.892252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:00.892530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:00.892812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] PATH_INFO: `/v3/users/ef0656ff3f504556b873fa9b26cce3db` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:00.894070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef0656ff3f504556b873fa9b26cce3db)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:00.896026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:00.958774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef0656ff3f504556b873fa9b26cce3db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:00.971873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef0656ff3f504556b873fa9b26cce3db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:00.983099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef0656ff3f504556b873fa9b26cce3db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:00.983924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef0656ff3f504556b873fa9b26cce3db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:00.984560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef0656ff3f504556b873fa9b26cce3db'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:01.022369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1069/2137] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:00 2022] DELETE /identity/v3/users/ef0656ff3f504556b873fa9b26cce3db => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:01.029979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:01.030480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:01.030933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:01.058743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:01.060528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:01.086065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:01.165743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1069/2138] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:01.175441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:01.178323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:01.301497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f66b7-5384-444d-abcd-26d2091d109c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:01.311447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e30ec-fe2f-4fba-b57f-803e3476b6e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:01.312527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e30ec-fe2f-4fba-b57f-803e3476b6e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:01.313062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e30ec-fe2f-4fba-b57f-803e3476b6e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:01.315176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e30ec-fe2f-4fba-b57f-803e3476b6e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:01.332388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e30ec-fe2f-4fba-b57f-803e3476b6e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:01.335384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e30ec-fe2f-4fba-b57f-803e3476b6e0 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:01.359490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1070/2139] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:15:01.839681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:01.843321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:01.887805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590235f-2964-460e-b876-7ed3ca24eeeb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:01.932759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:01.940566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:01.940566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:01.940566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] PATH_INFO: `/v3/projects/d01fe046dd914625a335524f05e3db1d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:01.941026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d01fe046dd914625a335524f05e3db1d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:01.946058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.049389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd01fe046dd914625a335524f05e3db1d', 'request_id': 'req-b52421ea-2d66-4f2e-ac91-12fdd0d05519'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:02.059897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd01fe046dd914625a335524f05e3db1d', 'request_id': 'req-b52421ea-2d66-4f2e-ac91-12fdd0d05519'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:02.070598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd01fe046dd914625a335524f05e3db1d', 'request_id': 'req-b52421ea-2d66-4f2e-ac91-12fdd0d05519'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:02.094404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] The token cache is being invalidate because project d01fe046dd914625a335524f05e3db1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:02.094915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:02.096047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-b52421ea-2d66-4f2e-ac91-12fdd0d05519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:02.098757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1070/2140] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:01 2022] DELETE /identity/v3/projects/d01fe046dd914625a335524f05e3db1d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:02.106470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.107137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.107781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.143380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:02.145368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:02.207579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:02.233868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1071/2141] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:02.246553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.246824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.247105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.270897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:02.271194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:02.284153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1071/2142] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:02.290668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.309251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca1cb2-0381-4e3c-aada-0a453c3c43cf None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.318196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.319379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.319962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.320543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.321993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:15:02.322623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.326407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.350191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1072/2143] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:15:02.359332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.371038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155db63-8277-4b0a-ac11-aa25411c92fc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.376742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.377579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.378064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.378375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.379162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.381578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.420666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1072/2144] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:02 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:02.425808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.447949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed631d0f-e102-4c6a-971f-ab1793570241 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.558693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.559534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.560475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.561550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.562617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.565443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.604051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1073/2145] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:02 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:02.611513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.630515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a7fe20-c84d-4c36-80ed-f7ff00c4e7b7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.643168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.644260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.644771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.645300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.646455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.650064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.672946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1073/2146] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:02.677760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.692888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a7f3a-59fa-44f6-8488-0483a3858c05 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.698954 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fdf50-7626-4a8c-8eb6-0efb69b1c53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.699601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fdf50-7626-4a8c-8eb6-0efb69b1c53c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.699885 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fdf50-7626-4a8c-8eb6-0efb69b1c53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.700325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fdf50-7626-4a8c-8eb6-0efb69b1c53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.702614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fdf50-7626-4a8c-8eb6-0efb69b1c53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.704417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fdf50-7626-4a8c-8eb6-0efb69b1c53c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.721865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1074/2147] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:02.728445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.740317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6695dd-47de-450f-a290-8e3f1b6be878 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.746094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.746817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.747166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.748104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] PATH_INFO: `/v3/projects/d379bf9c35444978b63c8a5a1e337928/users/9d951f022d244c348e045966c33d5663/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.767988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d379bf9c35444978b63c8a5a1e337928, user_id=9d951f022d244c348e045966c33d5663, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.770781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:02.786673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1074/2148] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:02 2022] PUT /identity/v3/projects/d379bf9c35444978b63c8a5a1e337928/users/9d951f022d244c348e045966c33d5663/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:02.792324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60ff75-4628-46c8-b2bf-c3a491c1a5d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.792740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60ff75-4628-46c8-b2bf-c3a491c1a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.793158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60ff75-4628-46c8-b2bf-c3a491c1a5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.815315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7c60ff75-4628-46c8-b2bf-c3a491c1a5d8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:02.816616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7c60ff75-4628-46c8-b2bf-c3a491c1a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:02.895516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1075/2149] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:02.905782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:02.907766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:02.961882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:02.975202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92064b1-d6a5-4030-a155-6a26bc9fbd4c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:02.979787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:02.980441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:02.980723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:02.981020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:02.992738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:02.994309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:03.006419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1075/2150] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:15:03.778019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b125-97ce-4f3c-a1f8-767ef4182d94 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:03.779074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b125-97ce-4f3c-a1f8-767ef4182d94 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:03.779074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b125-97ce-4f3c-a1f8-767ef4182d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:03.812074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6e32b125-97ce-4f3c-a1f8-767ef4182d94 None None] MFA Rules not processed for user `9d951f022d244c348e045966c33d5663`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:03.814070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6e32b125-97ce-4f3c-a1f8-767ef4182d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:03.863102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:03.916760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1076/2151] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:03.930125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:03.933503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:04.055615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d95da0-8b13-48e8-877c-79c5821a4aa0 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:04.062860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:04.063860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:04.064291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:04.064727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:04.082482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:04.084426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:04.098404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1076/2152] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:05.738642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd04b1-94f5-46f3-a0a8-cf58b25a7108 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:05.739289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd04b1-94f5-46f3-a0a8-cf58b25a7108 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:05.739289 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd04b1-94f5-46f3-a0a8-cf58b25a7108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:05.762906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8edd04b1-94f5-46f3-a0a8-cf58b25a7108 None None] MFA Rules not processed for user `9d951f022d244c348e045966c33d5663`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:05.765254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8edd04b1-94f5-46f3-a0a8-cf58b25a7108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:05.779717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1077/2153] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:06.509799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:06.511463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:06.537946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84443b39-1781-4e68-9d64-6b5c9ee02cd8 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:06.564133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:06.566281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:06.567650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:06.568017 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] PATH_INFO: `/v3/users/d3cedd1201e54b35a0febaf5cc79cbaa` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:06.571701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3cedd1201e54b35a0febaf5cc79cbaa)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:06.577378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:06.681575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3cedd1201e54b35a0febaf5cc79cbaa'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:06.682489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3cedd1201e54b35a0febaf5cc79cbaa'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:06.701930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3cedd1201e54b35a0febaf5cc79cbaa'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:06.718889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3cedd1201e54b35a0febaf5cc79cbaa'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:06.741982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3cedd1201e54b35a0febaf5cc79cbaa'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:06.754893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1077/2154] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:06 2022] DELETE /identity/v3/users/d3cedd1201e54b35a0febaf5cc79cbaa => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:06.756869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:06.757366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:06.757526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:06.795600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:06.796919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:06.934469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1078/2155] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:06.946356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:06.948431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:07.037106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd3084-cf07-4d63-875a-ab2cfb255487 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:07.042569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:07.043396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:07.043678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:07.043951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:07.057177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:07.059383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:07.073114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1078/2156] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:07.470054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:07.471897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:07.485012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842034f-218b-4c47-90db-86c1e5302d80 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:07.490149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:07.490804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:07.491090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:07.491372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] PATH_INFO: `/v3/projects/b6b61376c7294b7f8ce3a9c269508ec5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:07.492559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b61376c7294b7f8ce3a9c269508ec5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:07.494550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:07.545480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b61376c7294b7f8ce3a9c269508ec5', 'request_id': 'req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.551832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b61376c7294b7f8ce3a9c269508ec5', 'request_id': 'req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.558788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b61376c7294b7f8ce3a9c269508ec5', 'request_id': 'req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.571698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] The token cache is being invalidate because project b6b61376c7294b7f8ce3a9c269508ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:07.571990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.572624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.574192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1079/2157] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:07 2022] DELETE /identity/v3/projects/b6b61376c7294b7f8ce3a9c269508ec5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:07.579390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:07.586905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:07.591992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c82c13-bd2d-4881-9cde-ea6967f1bbce service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:07.662326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:07.662989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:07.663265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:07.663550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] PATH_INFO: `/v3/users/6d199b85be7542b2ad3c5f03f82efddf` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:07.665038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d199b85be7542b2ad3c5f03f82efddf)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:07.667011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:07.742431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d199b85be7542b2ad3c5f03f82efddf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.743751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d199b85be7542b2ad3c5f03f82efddf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.766788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d199b85be7542b2ad3c5f03f82efddf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.778416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d199b85be7542b2ad3c5f03f82efddf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.789820 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-bc26c175-c716-47a3-91f6-e1713cd25ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d199b85be7542b2ad3c5f03f82efddf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:07.798378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1079/2158] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:07 2022] DELETE /identity/v3/users/6d199b85be7542b2ad3c5f03f82efddf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:08.265531 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.267736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:08.304406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c3fa9-ac15-441a-ae0d-6a1ad5143a89 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.407110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:08.416101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.417465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.418105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.418669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] PATH_INFO: `/v3/projects/5cdc5dd317254b38801d0dfb4930617b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.421404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cdc5dd317254b38801d0dfb4930617b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:08.423763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:08.484494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cdc5dd317254b38801d0dfb4930617b', 'request_id': 'req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:08.493100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cdc5dd317254b38801d0dfb4930617b', 'request_id': 'req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:08.500116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cdc5dd317254b38801d0dfb4930617b', 'request_id': 'req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:08.516640 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] The token cache is being invalidate because project 5cdc5dd317254b38801d0dfb4930617b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:08.516933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:08.517686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c51ef786-2a0c-464d-9e6a-1f5d4a7461f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:08.519552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1080/2159] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:08 2022] DELETE /identity/v3/projects/5cdc5dd317254b38801d0dfb4930617b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:08.524601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.524888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.525197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.545556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:08.546783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:08.613091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1080/2160] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:08.621821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.623051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.623507 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.649519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:08.650759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:08.661035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1081/2161] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:08.666208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.681048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e42b-6084-4382-a3cc-2ff51df016a7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.688439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.689299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.689584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.689941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.690684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:15:08.690993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:08.692605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:08.706126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1081/2162] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:15:08.710749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.720835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97767f9-77d4-4ec6-8859-57f6395759ef None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.726147 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.726801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.727077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.727348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.728032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:08.729992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:08.763899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1082/2163] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:08 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:08.768350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.778641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0376d64-40fe-457e-a663-a05cfba5efa1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.787698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:08.841540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.842280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.842562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.842837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.843514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:08.845496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:08.875294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1082/2164] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:08 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:08.877718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.892309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e018f9d-9d11-4189-9357-ab08f7e52dda admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.897330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.897968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.898249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.898527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.899239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:08.901087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:08.913455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1083/2165] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:08.917740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.936652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf6f77-bcb5-4073-8872-a1268fe0b23c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.942784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976408-6050-4c57-a16a-6df7b6595b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.943432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976408-6050-4c57-a16a-6df7b6595b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.943720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976408-6050-4c57-a16a-6df7b6595b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.943985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976408-6050-4c57-a16a-6df7b6595b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:08.944704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d976408-6050-4c57-a16a-6df7b6595b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:08.946417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d976408-6050-4c57-a16a-6df7b6595b01 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:08.959918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1083/2166] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:15:08.965891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:08.978419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50851a4d-d880-4096-a8ad-319319fba345 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:08.984627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:08.985463 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:08.985823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:08.986194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] PATH_INFO: `/v3/projects/00d6a276e8f24929a3aae2fdda0b3e58/users/3affeeddbacf435ab0e0fcd4a2f042ca/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.007708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d6a276e8f24929a3aae2fdda0b3e58, user_id=3affeeddbacf435ab0e0fcd4a2f042ca, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.014251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.039367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1084/2167] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:08 2022] PUT /identity/v3/projects/00d6a276e8f24929a3aae2fdda0b3e58/users/3affeeddbacf435ab0e0fcd4a2f042ca/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:09.052501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.052846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.053222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.095971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] MFA Rules not processed for user `3affeeddbacf435ab0e0fcd4a2f042ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:09.097563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:09.196784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1084/2168] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:09.201953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.287519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae772ce-714f-4ab9-8979-49e8b448264d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:09.296467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:09.297483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.297922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.298344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.299816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.303502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.357646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1085/2169] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:09 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:15:09.361840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.373936 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94231028-f65f-4728-9dda-fcb84b6dfabf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:09.439144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:09.439780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.440049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.440453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.441083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.443178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.468550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1085/2170] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:09 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:09.472668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.484075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b3dc2-e094-4d4b-abfc-b65cd334e976 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:09.488910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:09.489580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.489859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.490189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.491037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.493308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.494969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:09.514983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1086/2171] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:09.515086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.528343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2b748-a0a5-4b25-98f3-1b37ab6bbf8d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:09.533868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164bec3a-630d-4ebe-aadc-d9d51acab7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:09.534557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bec3a-630d-4ebe-aadc-d9d51acab7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.534922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bec3a-630d-4ebe-aadc-d9d51acab7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.535203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164bec3a-630d-4ebe-aadc-d9d51acab7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.541625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bec3a-630d-4ebe-aadc-d9d51acab7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.543295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164bec3a-630d-4ebe-aadc-d9d51acab7d3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.546385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:09.576635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.577190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1086/2172] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:15:09.590062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8e6cc-2eab-4bea-a05a-8db696da01d0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:09.595452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:09.596233 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.596592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.597114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] PATH_INFO: `/v3/projects/0cef088b69684a95a0702b64e7eb54ba/users/0f688dd1a33348489d092dd63cb6a086/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.617891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cef088b69684a95a0702b64e7eb54ba, user_id=0f688dd1a33348489d092dd63cb6a086, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.621099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.649856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1087/2173] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:09 2022] PUT /identity/v3/projects/0cef088b69684a95a0702b64e7eb54ba/users/0f688dd1a33348489d092dd63cb6a086/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:09.663016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.663532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.663532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.704380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] MFA Rules not processed for user `0f688dd1a33348489d092dd63cb6a086`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:09.707469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:09.834930 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1087/2174] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:09.839916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.934170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33bcb-c5c9-4559-aee3-a079ca7175a2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:09.939735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:09.940375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:09.940658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:09.940933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:09.941707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:09.943583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:09.975065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1088/2175] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:09 2022] POST /identity/v3/projects => generated 387 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:09.979813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:09.992295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe4cc3-ffe3-46ad-89f2-82c5386dde30 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:10.135685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:10.136392 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.136746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.140418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.141323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:10.143362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:10.188324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:10.188947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1088/2176] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:09 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:10.212750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee9c6b-75ea-4fca-aa10-81199c40c453 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:10.223580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:10.227193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.227911 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.228643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.229937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:10.232782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:10.255308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1089/2177] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:15:10.261001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:10.278109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b6c2f-5926-44ce-8136-d3d309bb4aa7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:10.285117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:10.285839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.286121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.286396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] PATH_INFO: `/v3/projects/a371778191224052b98c5bdcbd1545d3/users/15ca042a62cd453aab15fda73331369f/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.318521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a371778191224052b98c5bdcbd1545d3, user_id=15ca042a62cd453aab15fda73331369f, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:10.326395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:10.384893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1089/2178] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:10 2022] PUT /identity/v3/projects/a371778191224052b98c5bdcbd1545d3/users/15ca042a62cd453aab15fda73331369f/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:10.390579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:10.411040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9b5dd-0c0f-4722-9938-2ac45bd37acd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:10.525105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca450eb-c912-4595-aa14-af662545a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:10.525841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca450eb-c912-4595-aa14-af662545a2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.526124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca450eb-c912-4595-aa14-af662545a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.526398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca450eb-c912-4595-aa14-af662545a2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.527191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca450eb-c912-4595-aa14-af662545a2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:10.529697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca450eb-c912-4595-aa14-af662545a2c0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:10.541467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1090/2179] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:15:10.547376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:10.558976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134936fe-1263-4007-acfa-a4b657ac28f0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:10.564412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:10.565299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.565794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.566149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] PATH_INFO: `/v3/projects/a371778191224052b98c5bdcbd1545d3/users/15ca042a62cd453aab15fda73331369f/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.568423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a371778191224052b98c5bdcbd1545d3, user_id=15ca042a62cd453aab15fda73331369f, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:10.571322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:10.584550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1090/2180] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:10 2022] PUT /identity/v3/projects/a371778191224052b98c5bdcbd1545d3/users/15ca042a62cd453aab15fda73331369f/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:10.596126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.596655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.596944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.630855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] MFA Rules not processed for user `15ca042a62cd453aab15fda73331369f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:10.632290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:10.676317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:10.750365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1091/2181] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:10.761891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:10.764259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:10.830270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:10.849492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4a358-0208-4c0f-bb61-86aa2c9381ee admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:10.853912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:10.854581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:10.854916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:10.855206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:10.867064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:10.868543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:10.880571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1091/2182] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:11.515430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:11.517076 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:11.531707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a876185-d5e8-46a9-9b78-cef5555c8539 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:11.536303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:11.536944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:11.537254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:11.537529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:11.617842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:11.619532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:11.633770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1092/2183] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:16.329436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:16.338709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df04c1d-f64d-45c9-a533-e8846023f71c service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:16.343008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5227f19-b8aa-4742-a76e-91db298e0641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:16.343637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227f19-b8aa-4742-a76e-91db298e0641 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:16.343918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227f19-b8aa-4742-a76e-91db298e0641 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:16.344195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5227f19-b8aa-4742-a76e-91db298e0641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:16.409830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5227f19-b8aa-4742-a76e-91db298e0641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:16.412437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5227f19-b8aa-4742-a76e-91db298e0641 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:16.426004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1092/2184] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:19.371705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:19.460445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc70157-11da-46ea-bb33-268ddec37d64 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:19.465817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:19.467448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:19.467946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:19.468355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] PATH_INFO: `/v3/users/3affeeddbacf435ab0e0fcd4a2f042ca` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:19.469832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3affeeddbacf435ab0e0fcd4a2f042ca)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:19.472128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:19.528332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:19.531084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:19.531632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:19.535457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3affeeddbacf435ab0e0fcd4a2f042ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:19.543692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3affeeddbacf435ab0e0fcd4a2f042ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:19.551645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3affeeddbacf435ab0e0fcd4a2f042ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:19.552321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3affeeddbacf435ab0e0fcd4a2f042ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:19.552990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c8cc9bae-e17f-4110-9ade-c37d1b20bfc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3affeeddbacf435ab0e0fcd4a2f042ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:19.559650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:19.565696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] MFA Rules not processed for user `c6291932ceb84b03924c0f056f29c2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:19.567984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:19.582040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1093/2185] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:19 2022] DELETE /identity/v3/users/3affeeddbacf435ab0e0fcd4a2f042ca => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:19.586639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:19.586940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:19.587198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:19.608476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:19.609875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:19.666942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1093/2186] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:19.685610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1094/2187] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:19.706816 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:19.708306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:19.804879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba086a5-7d6f-43c2-b4a2-7894a460af16 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:19.881180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:19.899353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:19.899604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:19.899882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:19.900342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:19.917663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:19.921562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:19.941289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1094/2188] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:20.581732 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:20.583391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:20.600307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebe6de-8a9b-4816-ad19-3dd567753ea3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:20.606908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:20.607906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:20.608497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:20.610012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] PATH_INFO: `/v3/projects/00d6a276e8f24929a3aae2fdda0b3e58` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:20.612428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d6a276e8f24929a3aae2fdda0b3e58)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:20.616632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:20.709363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d6a276e8f24929a3aae2fdda0b3e58', 'request_id': 'req-8e787499-912e-44e2-a5bb-6f30aaaffbcc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.723736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d6a276e8f24929a3aae2fdda0b3e58', 'request_id': 'req-8e787499-912e-44e2-a5bb-6f30aaaffbcc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.736135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d6a276e8f24929a3aae2fdda0b3e58', 'request_id': 'req-8e787499-912e-44e2-a5bb-6f30aaaffbcc'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.759286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] The token cache is being invalidate because project 00d6a276e8f24929a3aae2fdda0b3e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:20.759571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.761269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.763623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1095/2189] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:20 2022] DELETE /identity/v3/projects/00d6a276e8f24929a3aae2fdda0b3e58 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:20.767986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:20.782986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f845b37-7d33-446b-acab-717a58758c9b service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:20.873657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:20.874445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:20.874881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:20.875293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] PATH_INFO: `/v3/users/0f688dd1a33348489d092dd63cb6a086` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:20.876649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f688dd1a33348489d092dd63cb6a086)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:20.879156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:20.972565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f688dd1a33348489d092dd63cb6a086'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.973663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f688dd1a33348489d092dd63cb6a086'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:20.999742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f688dd1a33348489d092dd63cb6a086'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.009969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f688dd1a33348489d092dd63cb6a086'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.026616 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f688dd1a33348489d092dd63cb6a086'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.046961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1095/2190] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:20 2022] DELETE /identity/v3/users/0f688dd1a33348489d092dd63cb6a086 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:21.520745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:21.522326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:21.540432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e787499-912e-44e2-a5bb-6f30aaaffbcc admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:21.666669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:21.670016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:21.670778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:21.671488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] PATH_INFO: `/v3/projects/0cef088b69684a95a0702b64e7eb54ba` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:21.673627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cef088b69684a95a0702b64e7eb54ba)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:21.676122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:21.705473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:21.769018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cef088b69684a95a0702b64e7eb54ba', 'request_id': 'req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.778389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cef088b69684a95a0702b64e7eb54ba', 'request_id': 'req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.798962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cef088b69684a95a0702b64e7eb54ba', 'request_id': 'req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.819706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] The token cache is being invalidate because project 0cef088b69684a95a0702b64e7eb54ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:21.819981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.821327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:21.823343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1096/2191] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:21 2022] DELETE /identity/v3/projects/0cef088b69684a95a0702b64e7eb54ba => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:21.828089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:21.829636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:21.843655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc49fc-aaa8-4654-9a0e-8f30349307d4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:21.947983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:21.948565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:21.948848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:21.949153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] PATH_INFO: `/v3/users/15ca042a62cd453aab15fda73331369f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:21.950544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ca042a62cd453aab15fda73331369f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:21.952524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:21.959917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:22.021986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ca042a62cd453aab15fda73331369f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.022845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ca042a62cd453aab15fda73331369f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.041282 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ca042a62cd453aab15fda73331369f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.053027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ca042a62cd453aab15fda73331369f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.062702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d6aad4b8-86d2-4f63-809a-de7870f3af34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ca042a62cd453aab15fda73331369f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.078908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1096/2192] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:21 2022] DELETE /identity/v3/users/15ca042a62cd453aab15fda73331369f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:22.434287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:22.437805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:22.458691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7111b-bb20-45f0-b8b9-3a1c590c346c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:22.543795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:22.545427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:22.546003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:22.546376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] PATH_INFO: `/v3/projects/a371778191224052b98c5bdcbd1545d3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:22.548055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a371778191224052b98c5bdcbd1545d3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:22.550508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:22.630339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a371778191224052b98c5bdcbd1545d3', 'request_id': 'req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.640156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a371778191224052b98c5bdcbd1545d3', 'request_id': 'req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.646860 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a371778191224052b98c5bdcbd1545d3', 'request_id': 'req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.662971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] The token cache is being invalidate because project a371778191224052b98c5bdcbd1545d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:22.663265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.663927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0ca7363b-ee04-4494-8ef3-8cc88b2ca321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:22.665521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1097/2193] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:22 2022] DELETE /identity/v3/projects/a371778191224052b98c5bdcbd1545d3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:22.672637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:22.673019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:22.673352 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:22.695106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:22.696570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:22.784873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1097/2194] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:22.796053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:22.796635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:22.796635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:22.820753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:22.822049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:22.833592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1098/2195] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:22.838166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:22.850092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8bb28-42c9-443b-899d-ad6a5f133a09 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:22.856135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:22.857333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:22.857816 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:22.858513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:22.858821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:15:22.859125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:22.860806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:22.884351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1098/2196] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:15:22.889010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:22.903302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e870ee-f6c6-40b6-bf66-e982bb4aff8a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:22.909663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:22.910635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:22.910974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:22.911252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:22.912596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:22.916344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:22.953197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:22.973387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1099/2197] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:22 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:22.973919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:22.994536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8f53-2b0c-422e-ac4f-880f38193e89 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:23.148781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:23.150122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.150648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.151288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.152576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:23.159583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:23.255183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:23.281782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1099/2198] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:22 2022] POST /identity/v3/users => generated 286 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:23.285502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:23.315591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30608b6-6dac-41d8-b899-40f993d3f678 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:23.326111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:23.327218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.327613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.328017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.329058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:23.331502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:23.350675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1100/2199] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:23.358499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:23.383748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c133e63-eb01-4144-a708-dba068e8eb63 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:23.393262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b659dc-e557-4be5-b7a4-6ed4dbd00474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:23.394110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b659dc-e557-4be5-b7a4-6ed4dbd00474 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.394487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b659dc-e557-4be5-b7a4-6ed4dbd00474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.394996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b659dc-e557-4be5-b7a4-6ed4dbd00474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.396341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b659dc-e557-4be5-b7a4-6ed4dbd00474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:23.403097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b659dc-e557-4be5-b7a4-6ed4dbd00474 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:23.417422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1100/2200] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:23.423216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:23.437636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bce5c-f7db-47a5-9da5-6bc24fee9873 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:23.444037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:23.444772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.445106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.445390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] PATH_INFO: `/v3/projects/556e494e04e5439db6b644e437c23ac1/users/89b1d6261502424389cbcf4121cf6ca6/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.463292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556e494e04e5439db6b644e437c23ac1, user_id=89b1d6261502424389cbcf4121cf6ca6, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:23.466789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:23.483457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1101/2201] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:23 2022] PUT /identity/v3/projects/556e494e04e5439db6b644e437c23ac1/users/89b1d6261502424389cbcf4121cf6ca6/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:23.488663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd38e90-67d5-4928-b7af-d49a51aadff1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.488971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd38e90-67d5-4928-b7af-d49a51aadff1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.489295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd38e90-67d5-4928-b7af-d49a51aadff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.510057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0cd38e90-67d5-4928-b7af-d49a51aadff1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:23.511181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0cd38e90-67d5-4928-b7af-d49a51aadff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:23.587464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1101/2202] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:23.599050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:23.600874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:23.636791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001512e-6fa8-4da5-b70d-f098415e5d72 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.636791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001512e-6fa8-4da5-b70d-f098415e5d72 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.637111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001512e-6fa8-4da5-b70d-f098415e5d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.658318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7001512e-6fa8-4da5-b70d-f098415e5d72 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:23.659554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7001512e-6fa8-4da5-b70d-f098415e5d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:23.688608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d481c-049f-472f-83e0-5de254e0a810 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:23.693128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:23.693778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.694067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.694350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.706082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:23.707762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:23.720178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1102/2203] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:15:23.732770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1102/2204] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:15:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:23.802426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:23.804388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:23.943456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee201-d4db-491c-9cdd-0fbd3d046436 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:23.961122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:23.964378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:23.964378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:23.965171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:23.990680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:23.997326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:24.037312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1103/2205] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:15:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 17:15:25.414697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:25.415181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:25.415669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:25.509473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] MFA Rules not processed for user `89b1d6261502424389cbcf4121cf6ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:25.511264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:25.646464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1103/2206] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:25.651326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:25.673206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a652c41-5bbc-4535-8093-dbfe196b1578 service swift] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:25.688558 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:25.689659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:25.690225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:25.690849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:25.691912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:25.699030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:25.765177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1104/2207] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:25 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:15:25.769044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:25.788503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f4dbc-7fc3-45da-8211-368230fed41d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:25.838012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:25.989145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:25.989823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:25.990105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:25.990380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:25.991827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:25.993998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:26.121720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:26.123709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1104/2208] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:25 2022] POST /identity/v3/users => generated 293 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:15:26.157080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3ec3b-2c2c-4879-bc80-4dffaa36c66f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:26.180095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:26.181968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:26.185452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:26.186178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:26.189598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:26.199813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:26.232301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1105/2209] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:26.236919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:26.269576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105d4a6-fc22-44bf-882b-00aa0e764087 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:26.296945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:26.299761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:26.300348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:26.301459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] PATH_INFO: `/v3/projects/6cee341d2b774b81a6c091532e7ca908/users/f458b661d5484d38a86c9fe2a1cecd99/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:26.479441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cee341d2b774b81a6c091532e7ca908, user_id=f458b661d5484d38a86c9fe2a1cecd99, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:26.519234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:26.567044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1105/2210] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:26 2022] PUT /identity/v3/projects/6cee341d2b774b81a6c091532e7ca908/users/f458b661d5484d38a86c9fe2a1cecd99/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:26.573734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:26.614971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49371f56-d63f-479d-b14d-f278d1a80ba8 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:26.665639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:26.791144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfe27b-0413-426b-8fe6-bcb397141a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:26.791936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe27b-0413-426b-8fe6-bcb397141a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:26.792317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe27b-0413-426b-8fe6-bcb397141a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:26.792687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfe27b-0413-426b-8fe6-bcb397141a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:26.796429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfe27b-0413-426b-8fe6-bcb397141a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:26.798924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfe27b-0413-426b-8fe6-bcb397141a2d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:26.823270 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1106/2211] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:26.828597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:26.841838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36170709-d470-4719-9d9a-00397292f9de admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:26.847298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:26.848287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:26.848735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:26.849210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] PATH_INFO: `/v3/projects/6cee341d2b774b81a6c091532e7ca908/users/f458b661d5484d38a86c9fe2a1cecd99/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:26.851684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cee341d2b774b81a6c091532e7ca908, user_id=f458b661d5484d38a86c9fe2a1cecd99, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:26.855095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:26.884365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1106/2212] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:26 2022] PUT /identity/v3/projects/6cee341d2b774b81a6c091532e7ca908/users/f458b661d5484d38a86c9fe2a1cecd99/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:29.974744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4daf1f-d96e-4a83-9025-29131a17f530 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:29.975625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4daf1f-d96e-4a83-9025-29131a17f530 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:29.976078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4daf1f-d96e-4a83-9025-29131a17f530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:30.081729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ed4daf1f-d96e-4a83-9025-29131a17f530 None None] MFA Rules not processed for user `f458b661d5484d38a86c9fe2a1cecd99`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:30.088671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ed4daf1f-d96e-4a83-9025-29131a17f530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:30.424324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1107/2213] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:30.454673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:30.456948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:30.696719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:30.697073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:30.697454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:30.723165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a02ab-f53b-45b0-8a07-b2209311025d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:30.747945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9b154-1d28-445a-bbb7-307c6752343f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:30.748615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b154-1d28-445a-bbb7-307c6752343f service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:30.749658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b154-1d28-445a-bbb7-307c6752343f service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:30.749658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b154-1d28-445a-bbb7-307c6752343f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:30.760547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] MFA Rules not processed for user `97115d07acc5464bb7b097039471fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:30.772725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:30.935381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1108/2214] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:30.973363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9b154-1d28-445a-bbb7-307c6752343f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:30.975154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9b154-1d28-445a-bbb7-307c6752343f service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:31.007291 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1107/2215] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:33.039236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:33.041053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:33.057171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46656f7-f7e7-4894-88ad-1b86f2a46419 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:33.062844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:33.063814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:33.064111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:33.064522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:33.085479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:33.087467 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:33.100988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1109/2216] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:15:40.992148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:43.094965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=83 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:52.579550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:52.580224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:52.580224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:52.604731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:52.606130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:52.696678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1108/2217] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:15:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:52.766513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:52.901246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:53.014927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eb34d-9b7f-43c2-9c48-b014909d8965 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:53.021892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:53.023108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:53.024198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:53.024820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:53.048803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ad32d-99a3-4d39-86d0-6ad9094d10e3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:53.050457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:53.052587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:53.058346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:53.059049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:53.059335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:53.059663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] PATH_INFO: `/v3/users/89b1d6261502424389cbcf4121cf6ca6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:53.061283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b1d6261502424389cbcf4121cf6ca6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:53.063671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:53.086414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1110/2218] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:15:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:15:53.174346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b1d6261502424389cbcf4121cf6ca6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:53.175348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b1d6261502424389cbcf4121cf6ca6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:53.192357 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b1d6261502424389cbcf4121cf6ca6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:53.205623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b1d6261502424389cbcf4121cf6ca6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:53.220527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b1d6261502424389cbcf4121cf6ca6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:53.239867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1109/2219] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:52 2022] DELETE /identity/v3/users/89b1d6261502424389cbcf4121cf6ca6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:53.790117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:53.819923 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98e0b-6ec5-4fe3-8776-c7b2056a219a service swift] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:53.978475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:53.979421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:53.980051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:53.980351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] PATH_INFO: `/v3/projects/556e494e04e5439db6b644e437c23ac1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:53.982503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=556e494e04e5439db6b644e437c23ac1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:53.984908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:54.129508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556e494e04e5439db6b644e437c23ac1', 'request_id': 'req-2386dbb4-ae30-406a-b240-9ee021889f1d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.147959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556e494e04e5439db6b644e437c23ac1', 'request_id': 'req-2386dbb4-ae30-406a-b240-9ee021889f1d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.171176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556e494e04e5439db6b644e437c23ac1', 'request_id': 'req-2386dbb4-ae30-406a-b240-9ee021889f1d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.189978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] The token cache is being invalidate because project 556e494e04e5439db6b644e437c23ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:54.190439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.193580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.203876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1111/2220] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:53 2022] DELETE /identity/v3/projects/556e494e04e5439db6b644e437c23ac1 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:54.205750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:54.207447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:54.244863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629d0be-5708-48ec-be17-2bfd9851b26d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:54.291036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:54.482315 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:54.488235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:54.492847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:54.497570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] PATH_INFO: `/v3/users/f458b661d5484d38a86c9fe2a1cecd99` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:54.500331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f458b661d5484d38a86c9fe2a1cecd99)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:54.516647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:54.721208 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f458b661d5484d38a86c9fe2a1cecd99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.725957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f458b661d5484d38a86c9fe2a1cecd99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.789823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f458b661d5484d38a86c9fe2a1cecd99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.817753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f458b661d5484d38a86c9fe2a1cecd99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.841350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-dcecb219-e88e-4409-b654-f9ecb6caaf54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f458b661d5484d38a86c9fe2a1cecd99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:54.859998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1110/2221] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:15:54 2022] DELETE /identity/v3/users/f458b661d5484d38a86c9fe2a1cecd99 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:15:55.827075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:55.830509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:55.886926 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386dbb4-ae30-406a-b240-9ee021889f1d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:55.967740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:56.159893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:56.160919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:56.164084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:56.164556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] PATH_INFO: `/v3/projects/6cee341d2b774b81a6c091532e7ca908` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:56.166221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cee341d2b774b81a6c091532e7ca908)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:56.173389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:56.291793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cee341d2b774b81a6c091532e7ca908', 'request_id': 'req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:56.302697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cee341d2b774b81a6c091532e7ca908', 'request_id': 'req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:56.312599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cee341d2b774b81a6c091532e7ca908', 'request_id': 'req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:56.379647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] The token cache is being invalidate because project 6cee341d2b774b81a6c091532e7ca908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:15:56.380500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:56.381516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7ccdf3eb-5163-4a74-931f-8dcba80f55f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:15:56.394529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:56.394529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:56.394529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:56.397448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1112/2222] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:15:55 2022] DELETE /identity/v3/projects/6cee341d2b774b81a6c091532e7ca908 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:56.452433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:56.455553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:56.586865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:56.587136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:56.587449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:56.608857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1111/2223] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:56.629244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:56.631095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:56.670147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:56.680873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1113/2224] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:15:56.685086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9b1d4-aad9-461e-96a9-b61453f267b1 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:56.690842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:56.691495 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:56.691956 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:56.692472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:56.693369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:15:56.693754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:56.695572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:56.726089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:56.727841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1112/2225] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:15:56.768570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4aa92-25ed-4b16-a4f9-d9d5587e84bc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:56.804492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:56.805497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:56.805813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:56.806314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:56.807582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:56.819996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:56.959326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1114/2226] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:15:56 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:56.963273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:57.003644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fbcc4-16e8-451b-81e7-e7b8814be46d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:57.149382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:57.265835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:57.266805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:57.267286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:57.267765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:57.269010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:57.272699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:57.330188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1113/2227] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:15:56 2022] POST /identity/v3/users => generated 293 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:15:57.335305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:57.366091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff87af5-5d13-4ab1-a945-fa47666d25fb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:57.380838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:57.381525 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:57.381812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:57.382089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:57.382808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:57.384468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:57.410400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1115/2228] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:57.414489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:57.427564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab8e97-7044-4191-8e3c-3e5814a97083 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:57.433730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a7a952-3f46-4144-88df-ba5c279737fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:57.435056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a952-3f46-4144-88df-ba5c279737fb admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:57.435220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a952-3f46-4144-88df-ba5c279737fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:57.435647 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7a952-3f46-4144-88df-ba5c279737fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:57.436624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7a952-3f46-4144-88df-ba5c279737fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:57.438285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a7a952-3f46-4144-88df-ba5c279737fb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:57.460022 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:57.467431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1114/2229] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:15:57.472496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73572f56-2780-4c30-a9d6-3b90be314f03 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:57.478706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:57.479915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:57.480480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:57.480806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] PATH_INFO: `/v3/projects/894d5852e2d04e26932bdab0c16bf711/users/1cd9a563530c4e88a46eb030050e4cc5/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:57.503600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894d5852e2d04e26932bdab0c16bf711, user_id=1cd9a563530c4e88a46eb030050e4cc5, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:57.506761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:57.539544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ff69-9612-492c-a3e4-f8433e07646f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:57.539915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ff69-9612-492c-a3e4-f8433e07646f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:57.540294 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0ff69-9612-492c-a3e4-f8433e07646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:57.543200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1116/2230] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:15:57 2022] PUT /identity/v3/projects/894d5852e2d04e26932bdab0c16bf711/users/1cd9a563530c4e88a46eb030050e4cc5/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:15:57.578344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9fc0ff69-9612-492c-a3e4-f8433e07646f None None] MFA Rules not processed for user `1cd9a563530c4e88a46eb030050e4cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:15:57.580118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9fc0ff69-9612-492c-a3e4-f8433e07646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:57.756324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1115/2231] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:15:57.782020 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:15:57.784414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:15:57.985036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:15:58.008897 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e476e-5f54-4dc1-b62f-bd0473d1fb96 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:15:58.016790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:15:58.017865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:15:58.018315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:15:58.018753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:15:58.041743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:15:58.043363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:15:58.081372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1117/2232] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:15:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:16:01.095027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:01.095632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:01.095632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:01.154275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:01.157751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:01.307698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1116/2233] 10.222.0.48 () {58 vars in 967 bytes} [Tue Jul 26 17:16:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:01.392279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:01.394366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:01.620357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c6c0-b04f-4052-a9e5-4a163c35ba56 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:01.635108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:01.636069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:01.636524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:01.637003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:01.647875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:01.650501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:01.662439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1118/2234] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:16:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:16:05.813052 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:05.815615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:05.913396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:05.971251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85718-86f8-4f6d-b0d7-1c2670ca3e3a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:05.986128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:05.986837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:05.987141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:05.987413 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] PATH_INFO: `/v3/users/8ce2731e0c8647c2a7ea29a0569d2a1d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:05.990338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce2731e0c8647c2a7ea29a0569d2a1d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:05.995498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:06.133176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce2731e0c8647c2a7ea29a0569d2a1d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:06.142274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce2731e0c8647c2a7ea29a0569d2a1d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:06.186845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce2731e0c8647c2a7ea29a0569d2a1d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:06.217258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce2731e0c8647c2a7ea29a0569d2a1d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:06.237275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-fc131a74-715c-4a5e-8f7b-502cc4177612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce2731e0c8647c2a7ea29a0569d2a1d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:06.281473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1117/2235] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:16:05 2022] DELETE /identity/v3/users/8ce2731e0c8647c2a7ea29a0569d2a1d => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:07.060661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:07.074074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d6182-2088-470a-997d-9365a3d6a159 service swift] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:07.327090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:07.328286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:07.328986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:07.329517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] PATH_INFO: `/v3/projects/396220daaf3f4a89aab4007f746b96ea` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:07.332708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=396220daaf3f4a89aab4007f746b96ea)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:07.341544 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:07.526367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396220daaf3f4a89aab4007f746b96ea', 'request_id': 'req-cd9e0450-2652-435e-a2a1-eba1080663ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:07.541871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396220daaf3f4a89aab4007f746b96ea', 'request_id': 'req-cd9e0450-2652-435e-a2a1-eba1080663ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:07.557658 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396220daaf3f4a89aab4007f746b96ea', 'request_id': 'req-cd9e0450-2652-435e-a2a1-eba1080663ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:07.596924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] The token cache is being invalidate because project 396220daaf3f4a89aab4007f746b96ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:16:07.597698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:07.599222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd9e0450-2652-435e-a2a1-eba1080663ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:07.608598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1119/2236] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:16:07 2022] DELETE /identity/v3/projects/396220daaf3f4a89aab4007f746b96ea => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:07.610062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:07.610872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:07.610872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:07.667709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:07.673639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:07.843426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1118/2237] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:16:07.864815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:07.865675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:07.866150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:07.909292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:07.911159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:07.921985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1120/2238] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:07.928201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:07.948124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a954b-676e-412d-bf6f-5e7af56d6f11 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:07.959197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:07.960307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:07.960830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:07.962145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:07.962968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:16:07.963547 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:07.968646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:07.995999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1119/2239] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:16:07.999433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.009778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:08.021122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820be0c4-f874-40e0-bda0-48834f6c22cd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:08.033730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:08.034839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.035401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.035886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.037294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:08.041164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:08.112330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1121/2240] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:16:07 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:16:08.119576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.160509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424ee12-1eae-4d18-9414-e27df4c8fcd8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:08.264657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:08.284851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:08.285710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.286431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.287068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.288213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:08.291764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:08.332283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1120/2241] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:16:08 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:16:08.339194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.376059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0b77f-670f-4894-a0e8-dec7264b5598 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:08.385543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:08.386626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.387111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.388135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.389648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:08.392546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:08.419486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1122/2242] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:16:08.425425 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.447853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a1959-ed2f-4f6a-9976-ff916a53542c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:08.460175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567a6b7-0b74-4bd1-9353-d7b4a8b3a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:08.462530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567a6b7-0b74-4bd1-9353-d7b4a8b3a51b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.463624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567a6b7-0b74-4bd1-9353-d7b4a8b3a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.464422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567a6b7-0b74-4bd1-9353-d7b4a8b3a51b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.466790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567a6b7-0b74-4bd1-9353-d7b4a8b3a51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:08.472568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567a6b7-0b74-4bd1-9353-d7b4a8b3a51b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:08.503227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1121/2243] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:16:08.504881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.517548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549808eb-013b-4edf-80de-daead63d4df6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:08.528783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:08.531109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.531109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.531109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] PATH_INFO: `/v3/projects/9cca5830c0c04c6888c0e5389a7817d7/users/d2bae0b385d34728b50e0b804bb16498/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.559863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cca5830c0c04c6888c0e5389a7817d7, user_id=d2bae0b385d34728b50e0b804bb16498, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:08.564813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:08.590329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1123/2244] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:16:08 2022] PUT /identity/v3/projects/9cca5830c0c04c6888c0e5389a7817d7/users/d2bae0b385d34728b50e0b804bb16498/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:08.612022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.612851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.612851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.672584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] MFA Rules not processed for user `d2bae0b385d34728b50e0b804bb16498`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:08.676336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:08.782510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.786253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:08.846752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1122/2245] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:08.867757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:08.870327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:08.973560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83c9d5-3f2c-4133-87c4-0474273e7b13 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:08.984345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:08.986283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:08.986956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:08.987487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] PATH_INFO: `/v3/users/1cd9a563530c4e88a46eb030050e4cc5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:08.990513 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd9a563530c4e88a46eb030050e4cc5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:08.995143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:09.076638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ba7d-e878-42a1-a2ba-754043fa7635 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:09.085845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888177e4-540b-4fc9-9cdb-ca3a73df0b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:09.088305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888177e4-540b-4fc9-9cdb-ca3a73df0b44 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:09.091528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888177e4-540b-4fc9-9cdb-ca3a73df0b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:09.092574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888177e4-540b-4fc9-9cdb-ca3a73df0b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:09.103884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd9a563530c4e88a46eb030050e4cc5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:09.115951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888177e4-540b-4fc9-9cdb-ca3a73df0b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:09.121576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888177e4-540b-4fc9-9cdb-ca3a73df0b44 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:09.126694 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd9a563530c4e88a46eb030050e4cc5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:09.144437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd9a563530c4e88a46eb030050e4cc5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:09.146126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd9a563530c4e88a46eb030050e4cc5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:09.147337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd9a563530c4e88a46eb030050e4cc5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:09.150140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1123/2246] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:16:09.216797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1124/2247] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:16:08 2022] DELETE /identity/v3/users/1cd9a563530c4e88a46eb030050e4cc5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:09.224656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:09.225405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:09.226186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:09.259448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:09.261963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:09.388380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1124/2248] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:09.397438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:09.400485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:09.461131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:09.527825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266f08a-ce50-436c-a1cf-c112ca547398 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:09.532937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b448f98-f63a-4234-8f8d-fd1bd473cf77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:09.533812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b448f98-f63a-4234-8f8d-fd1bd473cf77 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:09.534094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b448f98-f63a-4234-8f8d-fd1bd473cf77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:09.534364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b448f98-f63a-4234-8f8d-fd1bd473cf77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:09.546924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b448f98-f63a-4234-8f8d-fd1bd473cf77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:09.549146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b448f98-f63a-4234-8f8d-fd1bd473cf77 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:09.566504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1125/2249] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:16:10.139746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:10.141659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:10.152473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:10.162366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9836be-1b19-498a-8324-6a4f7d3cba39 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:10.171649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:10.173122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.173648 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.174808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] PATH_INFO: `/v3/projects/894d5852e2d04e26932bdab0c16bf711` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.177152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=894d5852e2d04e26932bdab0c16bf711)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:10.195084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:10.313259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894d5852e2d04e26932bdab0c16bf711', 'request_id': 'req-a684db78-dfa7-4122-93e9-d18ab5b24fa5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:10.333220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894d5852e2d04e26932bdab0c16bf711', 'request_id': 'req-a684db78-dfa7-4122-93e9-d18ab5b24fa5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:10.354568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894d5852e2d04e26932bdab0c16bf711', 'request_id': 'req-a684db78-dfa7-4122-93e9-d18ab5b24fa5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:10.379192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] The token cache is being invalidate because project 894d5852e2d04e26932bdab0c16bf711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:16:10.379470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:10.380498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a684db78-dfa7-4122-93e9-d18ab5b24fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:10.392262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1125/2250] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:16:10 2022] DELETE /identity/v3/projects/894d5852e2d04e26932bdab0c16bf711 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:10.412258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.413161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.413350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.482907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:10.484197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:10.583441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1126/2251] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:16:10.598699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.599165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.599586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.624880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:10.626359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:10.642731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1126/2252] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:10.647782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:10.663728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae34c8-065b-4732-8d45-ca4155fb1a97 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:10.672541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:10.673831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.674112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.674408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.675203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:16:10.675617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:10.677447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:10.696667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1127/2253] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:16:10.702520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:10.713073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7fbaf-9992-4fe5-b7c4-3a5dc51c6b93 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:10.718896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:10.719544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.719828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.720106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.720805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:10.722788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:10.768502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1127/2254] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:16:10 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:16:10.769992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:10.783016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d4a28-b25b-4145-9a21-d8288b88168f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:10.872359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:10.874393 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.874561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.874801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.875593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:10.877770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:10.904584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1128/2255] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:16:10 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:16:10.909419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:10.922687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e1d29f-95b6-4303-ae90-614bd87d2632 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:10.927846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:10.928491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.928780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.929125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.929847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:10.931838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:10.952293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1128/2256] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:16:10.958984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:10.979133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b1dbf-aea3-4c5c-83eb-939a7b38d384 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:10.991484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cadada-9279-4241-8b99-f22dd58c85ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:10.992163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cadada-9279-4241-8b99-f22dd58c85ed admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:10.992674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cadada-9279-4241-8b99-f22dd58c85ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:10.993236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cadada-9279-4241-8b99-f22dd58c85ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:10.996121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cadada-9279-4241-8b99-f22dd58c85ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:11.000369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cadada-9279-4241-8b99-f22dd58c85ed admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:11.030097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1129/2257] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:16:11.034238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:11.054484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f238d5-1527-4550-8d68-4b47abb7d0d0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:11.063168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:11.064159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:11.064726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:11.065125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] PATH_INFO: `/v3/projects/1330d07479fe4c6eb2b7208ad85ce41f/users/635129483b37439ab6198212b2025528/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:11.093938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1330d07479fe4c6eb2b7208ad85ce41f, user_id=635129483b37439ab6198212b2025528, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:11.098171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:11.130376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1129/2258] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:16:11 2022] PUT /identity/v3/projects/1330d07479fe4c6eb2b7208ad85ce41f/users/635129483b37439ab6198212b2025528/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:11.155292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:11.155678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:11.156181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:11.206801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] MFA Rules not processed for user `635129483b37439ab6198212b2025528`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:11.208072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:11.235184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:11.313418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1130/2259] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:11.326473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:11.330501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:11.459811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078aee0-6503-48d6-b6a9-2f085ca52844 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:11.466389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36673b8-36b6-474b-9235-8b50324e665c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:11.467532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36673b8-36b6-474b-9235-8b50324e665c service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:11.468033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36673b8-36b6-474b-9235-8b50324e665c service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:11.468645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36673b8-36b6-474b-9235-8b50324e665c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:11.485417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36673b8-36b6-474b-9235-8b50324e665c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:11.488034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36673b8-36b6-474b-9235-8b50324e665c service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:11.505387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1130/2260] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:16:12.332452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:12.335046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:12.474352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2929349-bd50-4240-af24-03c3de6e80a1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:12.485752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:12.487649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:12.487649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:12.487935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:12.491277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:12.511928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:12.515014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b602e91-2edb-41e3-a353-faf592939025 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:12.536304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1131/2261] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:16:15.081213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678dddd-c759-4633-a057-c2f32b6eb5e7 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.081771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678dddd-c759-4633-a057-c2f32b6eb5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.082009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678dddd-c759-4633-a057-c2f32b6eb5e7 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.083696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1131/2262] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:16:15.092647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861acb5e-df33-45a1-92d5-f8561dee6623 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.093578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861acb5e-df33-45a1-92d5-f8561dee6623 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.093578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861acb5e-df33-45a1-92d5-f8561dee6623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.119596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-861acb5e-df33-45a1-92d5-f8561dee6623 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:15.121168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-861acb5e-df33-45a1-92d5-f8561dee6623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:15.208433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1132/2263] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:16:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:16:15.230736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678dddd-c759-4633-a057-c2f32b6eb5e7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:15.233455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1678dddd-c759-4633-a057-c2f32b6eb5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:15.265248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:15.339412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678dddd-c759-4633-a057-c2f32b6eb5e7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:15.345921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230b86c-4051-4ee5-b053-a22f16d6910b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:15.346990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b86c-4051-4ee5-b053-a22f16d6910b service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.347499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b86c-4051-4ee5-b053-a22f16d6910b service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.347991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230b86c-4051-4ee5-b053-a22f16d6910b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.366550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230b86c-4051-4ee5-b053-a22f16d6910b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:15.369389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230b86c-4051-4ee5-b053-a22f16d6910b service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:15.390858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1132/2264] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:16:15.488626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1799af-5d19-413f-8a89-af57b19dfb39 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.489010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1799af-5d19-413f-8a89-af57b19dfb39 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.489235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1799af-5d19-413f-8a89-af57b19dfb39 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.490618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1133/2265] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:16:15.498978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8e9e-306c-408c-abb8-192b21debcbd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.499407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8e9e-306c-408c-abb8-192b21debcbd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.499811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8e9e-306c-408c-abb8-192b21debcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.526473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-353d8e9e-306c-408c-abb8-192b21debcbd None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:15.529409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-353d8e9e-306c-408c-abb8-192b21debcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:15.614029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1133/2266] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:16:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:15.627273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1799af-5d19-413f-8a89-af57b19dfb39 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:15.629083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ed1799af-5d19-413f-8a89-af57b19dfb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:15.643907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1799af-5d19-413f-8a89-af57b19dfb39 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:15.648228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2b946-bdc2-414b-833c-a452baf87076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:15.648854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2b946-bdc2-414b-833c-a452baf87076 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.649170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2b946-bdc2-414b-833c-a452baf87076 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.649440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2b946-bdc2-414b-833c-a452baf87076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.658867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2b946-bdc2-414b-833c-a452baf87076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:15.660371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a2b946-bdc2-414b-833c-a452baf87076 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:15.669925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1134/2267] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:16:15.757225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0c214-cbe6-4684-addb-f0d1b2dd3d3e None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.757510 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0c214-cbe6-4684-addb-f0d1b2dd3d3e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.757787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0c214-cbe6-4684-addb-f0d1b2dd3d3e None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.759212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1134/2268] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 17:16:15.765870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b767bd-0523-47e5-a828-b3f821a24dd0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.766409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b767bd-0523-47e5-a828-b3f821a24dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.766829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b767bd-0523-47e5-a828-b3f821a24dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.797290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-82b767bd-0523-47e5-a828-b3f821a24dd0 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:15.799450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-82b767bd-0523-47e5-a828-b3f821a24dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:15.816721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1135/2269] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:16:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:15.828765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0c214-cbe6-4684-addb-f0d1b2dd3d3e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:15.830845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-10d0c214-cbe6-4684-addb-f0d1b2dd3d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:15.845765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0c214-cbe6-4684-addb-f0d1b2dd3d3e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:15.850673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:15.851409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:15.851780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:15.852144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:15.865577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:15.866938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:15.892295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1135/2270] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:16:25.788672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:25.876371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=53 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:33.936112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:33.936662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:33.937099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:33.969000 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:33.970731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:34.064600 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1136/2271] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:16:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:34.091198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:34.092496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:34.205274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a0db5-a45d-46e0-8e05-f7c42c4b39db service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:34.210609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd1ea4-a9e2-4327-9cf1-77bc4bb2c7d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:34.211420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd1ea4-a9e2-4327-9cf1-77bc4bb2c7d3 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:34.211702 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd1ea4-a9e2-4327-9cf1-77bc4bb2c7d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:34.211974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd1ea4-a9e2-4327-9cf1-77bc4bb2c7d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:34.221951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd1ea4-a9e2-4327-9cf1-77bc4bb2c7d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:34.223609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd1ea4-a9e2-4327-9cf1-77bc4bb2c7d3 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:34.233794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1136/2272] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:16:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 17:16:44.053970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=25 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:44.229713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=32 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:51.818990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:51.924820 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d153239-8b4c-4d9c-86f1-fb330fc2c098 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:51.932124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:51.934533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:51.935045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:51.935755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] PATH_INFO: `/v3/users/635129483b37439ab6198212b2025528` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:51.940165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=635129483b37439ab6198212b2025528)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:51.950186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:52.028666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '635129483b37439ab6198212b2025528'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.039814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '635129483b37439ab6198212b2025528'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.052035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '635129483b37439ab6198212b2025528'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.053162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '635129483b37439ab6198212b2025528'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.054378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '635129483b37439ab6198212b2025528'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.085413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1137/2273] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:16:51 2022] DELETE /identity/v3/users/635129483b37439ab6198212b2025528 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:52.091128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:52.091989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:52.091989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:52.112440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:52.113655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:52.211807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1137/2274] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:52.222002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:52.223505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:52.307809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b7f2-c6d0-4de5-8a51-d9acca754342 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:52.312650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1510b6-54f9-4ed5-8e56-84d863e499ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:52.313349 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1510b6-54f9-4ed5-8e56-84d863e499ed service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:52.313630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1510b6-54f9-4ed5-8e56-84d863e499ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:52.313895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1510b6-54f9-4ed5-8e56-84d863e499ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:52.326392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1510b6-54f9-4ed5-8e56-84d863e499ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:52.328483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1510b6-54f9-4ed5-8e56-84d863e499ed service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:52.347090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1138/2275] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:16:52.779201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:52.781635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:52.800221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1b312-2554-47a8-b640-57bdb9bf96bb None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:52.808567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:52.808567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:52.808567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:52.808567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] PATH_INFO: `/v3/projects/1330d07479fe4c6eb2b7208ad85ce41f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:52.809955 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1330d07479fe4c6eb2b7208ad85ce41f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:52.812863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:52.874118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1330d07479fe4c6eb2b7208ad85ce41f', 'request_id': 'req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.883044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1330d07479fe4c6eb2b7208ad85ce41f', 'request_id': 'req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.893496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1330d07479fe4c6eb2b7208ad85ce41f', 'request_id': 'req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.906475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] The token cache is being invalidate because project 1330d07479fe4c6eb2b7208ad85ce41f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:16:52.906746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.907555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-d865a0bc-fda5-4b96-ae15-3b42f8a5745c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:16:52.909555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1138/2276] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:16:52 2022] DELETE /identity/v3/projects/1330d07479fe4c6eb2b7208ad85ce41f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:52.921981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:52.921981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:52.922682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:52.946644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:52.947883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:52.969287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:53.026488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1139/2277] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:53.100421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.101059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.101059 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.125006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:53.126253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:53.134566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1139/2278] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:53.140387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.159035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f08a49-5bc5-4d38-8ae3-910c9ff8d98f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.165227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.165872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.166178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.166410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.167358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:16:53.167478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.170297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.184749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1140/2279] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:16:53.188537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.198753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98181e6a-917b-4716-9265-4e43ad62f8a8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.204327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.204998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.205273 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.205549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.206230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.208156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.239486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1140/2280] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:16:53 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:16:53.244008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.258817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469710f-2b61-470b-a1f2-bd723820e6e9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.322390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.323169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.323536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.323825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.325116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.327267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.351817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1141/2281] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:16:53 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:16:53.356946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.368603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d0d671-65ca-4c6e-a65a-3ed8617b6b6f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.373938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.374872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.374872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.375106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.375824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.378546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.389669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1141/2282] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:16:53.394215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.407265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f8507-f19f-4767-bbf6-be8f0dde11f1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.413310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8920f8-b010-4c59-aa3a-cfb48ada34ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.414253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8920f8-b010-4c59-aa3a-cfb48ada34ad admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.414655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8920f8-b010-4c59-aa3a-cfb48ada34ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.415031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8920f8-b010-4c59-aa3a-cfb48ada34ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.415908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8920f8-b010-4c59-aa3a-cfb48ada34ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.417873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8920f8-b010-4c59-aa3a-cfb48ada34ad admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.433383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1142/2283] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:16:53.437321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.448871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622b1d7-05b8-4001-9ee1-e694358807c9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.453877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.454570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.454859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.455141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] PATH_INFO: `/v3/projects/0a543d5eb6ef45569e57965779e4753e/users/d84cca5fc64f46109e9e62394da12437/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.477242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a543d5eb6ef45569e57965779e4753e, user_id=d84cca5fc64f46109e9e62394da12437, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.480124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.495364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1142/2284] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:16:53 2022] PUT /identity/v3/projects/0a543d5eb6ef45569e57965779e4753e/users/d84cca5fc64f46109e9e62394da12437/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:16:53.501412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c8346-2484-473c-83c6-4a4f39777e02 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.501701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c8346-2484-473c-83c6-4a4f39777e02 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.501981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c8346-2484-473c-83c6-4a4f39777e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.528453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5e0c8346-2484-473c-83c6-4a4f39777e02 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:53.529735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5e0c8346-2484-473c-83c6-4a4f39777e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:53.598107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1143/2285] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:16:53.610735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:53.612578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:53.638746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:53.683500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b91db-9984-4b4c-a5f4-08ce1209d4cb admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:53.687822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:53.688511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:53.688839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:53.689249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:53.700284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:53.701813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:53.712849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1143/2286] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:16:59.756621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:59.757256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:59.757356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:59.801351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] MFA Rules not processed for user `d84cca5fc64f46109e9e62394da12437`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:16:59.803063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:16:59.827456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:16:59.892214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1144/2287] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:16:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:16:59.896865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:59.912365 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc51d14-fe81-4c7f-973e-8f880101a9cd service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:16:59.917343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:16:59.917996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:16:59.918285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:16:59.918559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:16:59.919244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:16:59.921204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:16:59.957217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1144/2288] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:16:59 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:16:59.961656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:16:59.975867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a475abb-3b80-48f1-b52a-7a1bb1c9172d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:00.057930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:00.058299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:00.058580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:00.059542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:00.060599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:00.062734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:00.088828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1145/2289] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:16:59 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:17:00.094629 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:00.106601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84c1c8-573f-47b6-92d1-254ae5fee2e1 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:00.111961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:00.112689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:00.112689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:00.113476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:00.114008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:00.116097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:00.138163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1145/2290] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:17:00.145460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:00.162526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d59f0-a74a-41aa-9d29-5e096771cf5f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:00.179106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:00.180175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:00.180669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:00.181207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] PATH_INFO: `/v3/projects/61dbe2e4c2744463b6d76b38a8e6d881/users/f2ce5a6cc58e456e8426968c4bfe3cfe/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:00.223010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=61dbe2e4c2744463b6d76b38a8e6d881, user_id=f2ce5a6cc58e456e8426968c4bfe3cfe, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:00.227078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:00.251264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1146/2291] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:17:00 2022] PUT /identity/v3/projects/61dbe2e4c2744463b6d76b38a8e6d881/users/f2ce5a6cc58e456e8426968c4bfe3cfe/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:00.256171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:00.268245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05da73-2c31-48d4-90b7-8c70bfd240b9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:00.379516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f24e1-c2cc-44e6-893a-9a18b2dbb209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:00.380175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f24e1-c2cc-44e6-893a-9a18b2dbb209 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:00.380459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f24e1-c2cc-44e6-893a-9a18b2dbb209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:00.380729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f24e1-c2cc-44e6-893a-9a18b2dbb209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:00.381563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f24e1-c2cc-44e6-893a-9a18b2dbb209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:00.383238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f24e1-c2cc-44e6-893a-9a18b2dbb209 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:00.394975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1146/2292] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:17:00.404886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:00.427845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f95b-1e9e-44eb-95f9-6bad92569c5f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:00.434105 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:00.435132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:00.435506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:00.435864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] PATH_INFO: `/v3/projects/61dbe2e4c2744463b6d76b38a8e6d881/users/f2ce5a6cc58e456e8426968c4bfe3cfe/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:00.438536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=61dbe2e4c2744463b6d76b38a8e6d881, user_id=f2ce5a6cc58e456e8426968c4bfe3cfe, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:00.441616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:00.466033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1147/2293] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:17:00 2022] PUT /identity/v3/projects/61dbe2e4c2744463b6d76b38a8e6d881/users/f2ce5a6cc58e456e8426968c4bfe3cfe/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:06.996528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437b885-bf00-461b-a857-eba78822c855 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:06.997489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437b885-bf00-461b-a857-eba78822c855 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:06.997489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437b885-bf00-461b-a857-eba78822c855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:07.033634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7437b885-bf00-461b-a857-eba78822c855 None None] MFA Rules not processed for user `f2ce5a6cc58e456e8426968c4bfe3cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:07.034761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7437b885-bf00-461b-a857-eba78822c855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:07.147447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1147/2294] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:07.163353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:07.165832 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:07.323273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0b263-67fd-4422-b78e-07e580d81cdd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:07.345325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc63e6-8db8-4a4c-9d19-241f61cb746d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:07.346430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc63e6-8db8-4a4c-9d19-241f61cb746d service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:07.346890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc63e6-8db8-4a4c-9d19-241f61cb746d service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:07.347382 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc63e6-8db8-4a4c-9d19-241f61cb746d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:07.361171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:07.448699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc63e6-8db8-4a4c-9d19-241f61cb746d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:07.451917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc63e6-8db8-4a4c-9d19-241f61cb746d service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:07.464533 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1148/2295] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:17:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:17:08.539607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f2a13-9665-4fa2-9b3b-9d0e27dadffb None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:08.542186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f2a13-9665-4fa2-9b3b-9d0e27dadffb None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:08.542738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f2a13-9665-4fa2-9b3b-9d0e27dadffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:08.587613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e53f2a13-9665-4fa2-9b3b-9d0e27dadffb None None] MFA Rules not processed for user `9d951f022d244c348e045966c33d5663`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:08.589521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e53f2a13-9665-4fa2-9b3b-9d0e27dadffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:08.608534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:08.678427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1148/2296] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:09.106559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:09.107591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:09.107591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:09.127377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] MFA Rules not processed for user `d84cca5fc64f46109e9e62394da12437`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:09.128631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:09.136029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1149/2297] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:17:09.529799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:09.531249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:09.531743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:09.557951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:09.559772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:09.715657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1149/2298] 10.222.0.48 () {58 vars in 967 bytes} [Tue Jul 26 17:17:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:09.735168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:09.737033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:09.944261 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d612c54-2300-4474-b5c2-6f231805fcc7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:09.960744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8953f-1aa1-4b35-ba92-f0cb35dd1805 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:09.961384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8953f-1aa1-4b35-ba92-f0cb35dd1805 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:09.961675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8953f-1aa1-4b35-ba92-f0cb35dd1805 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:09.961959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8953f-1aa1-4b35-ba92-f0cb35dd1805 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:09.991108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8953f-1aa1-4b35-ba92-f0cb35dd1805 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:09.994869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8953f-1aa1-4b35-ba92-f0cb35dd1805 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:10.017070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1150/2299] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:17:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:17:16.032095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:16.034736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:16.293695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa8230-0c19-4d1b-8696-f834cdf847cf None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:16.305395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:16.307350 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:16.308338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:16.309856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] PATH_INFO: `/v3/users/d2bae0b385d34728b50e0b804bb16498` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:16.312133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2bae0b385d34728b50e0b804bb16498)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:16.315860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:16.397250 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2bae0b385d34728b50e0b804bb16498'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:16.398029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2bae0b385d34728b50e0b804bb16498'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:16.422986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:16.425445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2bae0b385d34728b50e0b804bb16498'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:16.441903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2bae0b385d34728b50e0b804bb16498'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:16.458777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2bae0b385d34728b50e0b804bb16498'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:16.467569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1150/2300] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:17:16 2022] DELETE /identity/v3/users/d2bae0b385d34728b50e0b804bb16498 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:16.473443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:16.474236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:16.474236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:16.507144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:16.509420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:16.613992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1151/2301] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:16.625901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:16.628240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:16.756075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e643e-34f2-4cc2-ba73-8894015d14a9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:16.762658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bf2fe-6c66-44d8-8ea0-f56bfcc28bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:16.764536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf2fe-6c66-44d8-8ea0-f56bfcc28bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:16.765501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf2fe-6c66-44d8-8ea0-f56bfcc28bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:16.766287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf2fe-6c66-44d8-8ea0-f56bfcc28bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:16.790191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bf2fe-6c66-44d8-8ea0-f56bfcc28bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:16.799815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bf2fe-6c66-44d8-8ea0-f56bfcc28bb9 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:16.821121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1151/2302] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:17:17.302010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:17.303657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:17.334549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2110c1-19b8-498f-9f7d-d7726a4d2268 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:17.354566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:17.354566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.354566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.354566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] PATH_INFO: `/v3/projects/9cca5830c0c04c6888c0e5389a7817d7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.360171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cca5830c0c04c6888c0e5389a7817d7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:17.363703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:17.415325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:17.438470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cca5830c0c04c6888c0e5389a7817d7', 'request_id': 'req-8edb1207-35f9-4027-84c0-37f763ef738a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:17.446647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cca5830c0c04c6888c0e5389a7817d7', 'request_id': 'req-8edb1207-35f9-4027-84c0-37f763ef738a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:17.454615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cca5830c0c04c6888c0e5389a7817d7', 'request_id': 'req-8edb1207-35f9-4027-84c0-37f763ef738a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:17.473331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] The token cache is being invalidate because project 9cca5830c0c04c6888c0e5389a7817d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:17:17.473629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:17.474417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8edb1207-35f9-4027-84c0-37f763ef738a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:17:17.475902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1152/2303] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:17:17 2022] DELETE /identity/v3/projects/9cca5830c0c04c6888c0e5389a7817d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:17.482552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.483133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.483590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.505723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:17.506990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:17.575539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1152/2304] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:17.585470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.585682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.585938 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.608272 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:17.609580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:17.622881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1153/2305] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:17:17.628804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:17.642470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c94e7-622b-4984-98a6-29ae73706de4 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:17.648203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:17.649035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.649567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.649895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.650693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:17:17.651094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:17.654465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:17.675141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1153/2306] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:17:17.679846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:17.697211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d72897-fcb7-4244-9dd5-f7bf998c578d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:17.704163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:17.705479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.706104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.706490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.707379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:17.710215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:17.751359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1154/2307] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:17:17 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:17:17.756622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:17.773479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89940-b3e4-4ce7-9ca3-77cb52c5a7e3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:17.880813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:17.881490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.881768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.882046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.882715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:17.884666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:17.905251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:17.914153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1154/2308] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:17:17 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:17:17.920264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:17.939768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa71e88-68cb-40ba-a768-e08741df77f2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:17.948866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:17.950150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:17.950667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:17.951161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:17.952515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:17.955770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:17.976787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1155/2309] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:17:17.981659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:17.994532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8adc26-ff0d-4217-8e22-a5d75277081b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:18.002721 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3011b72-d2bf-4513-afe4-bd6a86c752de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:18.003755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3011b72-d2bf-4513-afe4-bd6a86c752de admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:18.004308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3011b72-d2bf-4513-afe4-bd6a86c752de admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:18.004605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3011b72-d2bf-4513-afe4-bd6a86c752de admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:18.005795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3011b72-d2bf-4513-afe4-bd6a86c752de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:18.007838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3011b72-d2bf-4513-afe4-bd6a86c752de admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:18.021625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1155/2310] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:17:18.026275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:18.039017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94b204-a472-4392-98cb-96e216c06c9f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:18.048692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:18.049942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:18.050452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:18.050951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] PATH_INFO: `/v3/projects/b888b1053bb043dbb88d674c9a504a86/users/dcf968aeb53c4215927642e9931b4a5e/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:18.081722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b888b1053bb043dbb88d674c9a504a86, user_id=dcf968aeb53c4215927642e9931b4a5e, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:18.085427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:18.102787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1156/2311] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:17:18 2022] PUT /identity/v3/projects/b888b1053bb043dbb88d674c9a504a86/users/dcf968aeb53c4215927642e9931b4a5e/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:18.111822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46515-fe93-411e-956a-eeceac2c53b5 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:18.112471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46515-fe93-411e-956a-eeceac2c53b5 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:18.113025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46515-fe93-411e-956a-eeceac2c53b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:18.145600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-06e46515-fe93-411e-956a-eeceac2c53b5 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:18.146948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-06e46515-fe93-411e-956a-eeceac2c53b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:18.261475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1156/2312] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:18.273704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:18.276431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:18.384098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a1a24-eebe-437a-b358-dda2d1fa64a6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:18.391267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:18.391934 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:18.392222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:18.392528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:18.406447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:18.407952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:18.425080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1157/2313] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:17:27.532258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:27.532861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:27.533429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:27.610046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] MFA Rules not processed for user `dcf968aeb53c4215927642e9931b4a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:27.611905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:27.713229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1157/2314] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:27.718592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:27.729008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786076-e87c-4d32-9022-b7bc9c97340b service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:27.735188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:27.735920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:27.736392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:27.736670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:27.737411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:27.739554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:27.771459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1158/2315] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:17:27 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:17:27.775812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:27.788097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df209-37b2-45e9-ac40-d1dfa0db8d4a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:27.877494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:27.877494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:27.877494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:27.877885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:27.878837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:27.881970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:27.913072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1158/2316] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:17:27 2022] POST /identity/v3/users => generated 296 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:17:27.914893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:27.931134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d17e0d-ab82-4fb8-bd46-a80d8d24ca47 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:27.939198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:27.939994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:27.940447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:27.941353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:27.942104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:27.943828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:27.961015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1159/2317] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:17:27.964485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:27.977576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff6005-aed6-4358-bfb0-d00be0a5289b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:27.983405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:27.984099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:27.984387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:27.984669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] PATH_INFO: `/v3/projects/f4b634e69a1b4a6d82b99a521aa639ef/users/aee33ea192294139ae964b8a96daf83b/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:27.989637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:28.012602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b634e69a1b4a6d82b99a521aa639ef, user_id=aee33ea192294139ae964b8a96daf83b, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:28.017632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:28.049118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1159/2318] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:17:27 2022] PUT /identity/v3/projects/f4b634e69a1b4a6d82b99a521aa639ef/users/aee33ea192294139ae964b8a96daf83b/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:28.050345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:28.062529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89568c1-4ac9-4309-8b52-09cc7902fe76 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:28.102932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:28.207057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95410c4-9675-4573-96b8-686f77867907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:28.207598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95410c4-9675-4573-96b8-686f77867907 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:28.208171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95410c4-9675-4573-96b8-686f77867907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:28.208171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95410c4-9675-4573-96b8-686f77867907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:28.208867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95410c4-9675-4573-96b8-686f77867907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:28.210590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95410c4-9675-4573-96b8-686f77867907 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:28.237305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:28.239240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1160/2319] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:17:28.269897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2b6c6-d89f-4880-bea1-60ea34ec5340 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:28.304048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbda14-ff0b-4ddd-8fbb-09d785ec3b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:28.305096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbda14-ff0b-4ddd-8fbb-09d785ec3b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:28.305575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbda14-ff0b-4ddd-8fbb-09d785ec3b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:28.306033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbda14-ff0b-4ddd-8fbb-09d785ec3b25 admin admin] PATH_INFO: `/v3/projects/f4b634e69a1b4a6d82b99a521aa639ef/users/aee33ea192294139ae964b8a96daf83b/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:28.311131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbda14-ff0b-4ddd-8fbb-09d785ec3b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b634e69a1b4a6d82b99a521aa639ef, user_id=aee33ea192294139ae964b8a96daf83b, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:28.319431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbda14-ff0b-4ddd-8fbb-09d785ec3b25 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:28.336224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1160/2320] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:17:28 2022] PUT /identity/v3/projects/f4b634e69a1b4a6d82b99a521aa639ef/users/aee33ea192294139ae964b8a96daf83b/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:17:31.218929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:31.219656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:31.219656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:31.280373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] MFA Rules not processed for user `c6291932ceb84b03924c0f056f29c2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:31.282132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:31.488780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1161/2321] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:17:38.331877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=11,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:38.474862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:38.475151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:38.475434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:38.514669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] MFA Rules not processed for user `aee33ea192294139ae964b8a96daf83b`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:17:38.515983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:38.614040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1161/2322] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:17:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:17:38.631030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:38.633186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:38.739710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca48cee-9bb9-402f-bbf9-99d1ae6ed29b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:38.745484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:38.746692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:38.746967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:38.747247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:38.815592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:38.817337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:38.821253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:38.828994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1162/2323] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:17:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:17:48.609303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=34 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:48.826072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:17:58.039130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:58.109113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758beeb5-fa0c-4509-ac6f-be2e808b9c68 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:58.113744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:58.114609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:58.115064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:58.115224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:58.117213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:58.133359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:58.134846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:58.149668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1162/2324] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:17:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:17:58.398803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:17:58.413376 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5edf5-80a4-4ad0-abd5-4f1d7bc4c1eb service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:17:58.422275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:17:58.423647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:17:58.423981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:17:58.424628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:17:58.427151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:17:58.532552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:17:58.534071 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d06b2-56dc-4d1d-acc3-e5eb77111bb4 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:17:58.550188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1163/2325] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:17:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:00.996429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:00.998164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:01.010045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179b8ff-47ec-46e7-a962-c7e87d8f18d5 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:01.016658 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:01.017356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:01.017801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:01.018091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:01.020152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:01.045458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:01.049935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5a196-ad3a-43ec-a59d-fa3752943dee service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:01.080837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1163/2326] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:03.726709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef548ad0-27f2-4ce7-8278-61546668b460 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:03.727551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef548ad0-27f2-4ce7-8278-61546668b460 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:03.727551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef548ad0-27f2-4ce7-8278-61546668b460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:03.754879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ef548ad0-27f2-4ce7-8278-61546668b460 None None] MFA Rules not processed for user `dcf968aeb53c4215927642e9931b4a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:18:03.756321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ef548ad0-27f2-4ce7-8278-61546668b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:03.782064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1164/2327] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:18:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:18:06.142421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:06.142925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:06.143033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:06.178698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] MFA Rules not processed for user `97115d07acc5464bb7b097039471fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:18:06.179951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:06.291087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1164/2328] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:18:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:18:13.746545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=33 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:18:16.282760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=57 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:18:26.520401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:26.521553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:26.521946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:26.550936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] MFA Rules not processed for user `ab248df5c41d4495b9c9f46ef33c1631`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:18:26.552783 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:26.625564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1165/2329] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:18:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:18:26.642589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:26.644306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:26.663103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb783836-9779-407a-9e7b-1024fbfcea0d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:26.671467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:26.672126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:26.672408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:26.672684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:26.675077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:26.689466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:26.691409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:26.704045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1165/2330] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:27.085807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:27.099049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ebe96-7bf4-4ce0-afee-feac756ed128 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:27.103999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:27.105333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:27.105712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:27.106253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:27.108746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:27.122975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:27.125271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:27.136879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1166/2331] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:27.361213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:27.377699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e8481-88c6-4fe3-aa60-6fd00fc9cf28 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:27.381832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:27.382462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:27.382747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:27.383023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:27.394434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:27.395968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:27.408873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1166/2332] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:18:27.576304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:27.578651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:27.720522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a922a-00b0-4038-8729-a3e77d52e4aa service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:27.728102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:27.729317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:27.730562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:27.730940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:27.733233 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:27.747249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:27.749488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:27.762246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1167/2333] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:18:29.880259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:29.899293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499eb916-114f-4f0a-803b-93c014e5fb0b service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:29.909428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:29.912032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:29.912373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:29.912654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:29.916198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:29.937395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:29.941089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:29.953095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1167/2334] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:31.702542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:31.705807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:31.801847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd2549-ec14-4769-bdbe-0432ccdc218b service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:31.806986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:31.807660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:31.808124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:31.808411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] PATH_INFO: `/v3/users/9d951f022d244c348e045966c33d5663` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:31.809678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d951f022d244c348e045966c33d5663)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:31.811696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:31.840499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:18:31.877546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d951f022d244c348e045966c33d5663'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:31.891589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d951f022d244c348e045966c33d5663'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:31.903749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d951f022d244c348e045966c33d5663'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:31.904839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d951f022d244c348e045966c33d5663'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:31.905291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d951f022d244c348e045966c33d5663'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:31.941081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1168/2335] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:18:31 2022] DELETE /identity/v3/users/9d951f022d244c348e045966c33d5663 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:18:32.293962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:32.318344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7428b58-da41-4133-981b-55595b20b373 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:32.475935 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:32.476759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:32.477304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:32.477847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] PATH_INFO: `/v3/projects/d379bf9c35444978b63c8a5a1e337928` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:32.479307 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d379bf9c35444978b63c8a5a1e337928)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:32.483603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:32.566560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd379bf9c35444978b63c8a5a1e337928', 'request_id': 'req-cd018663-b9d1-484a-975c-0373ed9136ed'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:32.584740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd379bf9c35444978b63c8a5a1e337928', 'request_id': 'req-cd018663-b9d1-484a-975c-0373ed9136ed'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:32.597221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd379bf9c35444978b63c8a5a1e337928', 'request_id': 'req-cd018663-b9d1-484a-975c-0373ed9136ed'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:32.613782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] The token cache is being invalidate because project d379bf9c35444978b63c8a5a1e337928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:18:32.614112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:32.615050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:18:32.617890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1168/2336] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:18:32 2022] DELETE /identity/v3/projects/d379bf9c35444978b63c8a5a1e337928 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:18:34.042937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:34.044534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:34.113279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941d846-76c9-4832-9df1-45987ef65971 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:34.186921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:34.187471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:34.187749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:34.188023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:34.190152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:34.259408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:34.261067 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d93bae-b267-47cc-b3da-b37bc47b04e5 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:34.272070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1169/2337] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:42.571539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:42.573391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:42.646326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:18:42.685857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd018663-b9d1-484a-975c-0373ed9136ed admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:42.691902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:42.692532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:42.692809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:42.693121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:42.695018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:42.708890 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:42.710543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:42.723334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1169/2338] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:44.267399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:18:45.234875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:45.235181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:45.235455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:45.257371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] MFA Rules not processed for user `459db7ff580948dc853b9f843700fe3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:18:45.258590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:45.266159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1170/2339] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:18:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:18:45.611492 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:45.613199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:45.676054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb70f16-57ce-42fc-b332-86fc77744ddd service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:45.682035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:45.683758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:45.684265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:45.684643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:45.686654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:45.697451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:45.699476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:45.709134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1170/2340] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:49.295699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:49.322521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241141c3-d17b-4702-96e8-21575c6061e8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:49.334826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:49.335991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:49.336561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:49.337179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:49.342123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:49.366710 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:49.368983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:49.384164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1171/2341] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:52.949700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:52.963551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9874c1-aed5-455b-a384-723bc66e4932 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:52.974651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:52.975317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:52.975598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:52.975874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:52.977803 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:53.055787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:53.058257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:53.069351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1171/2342] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:18:55.352229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:18:55.372197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb764f4a-8c83-4b5b-98e8-ade402b29ba5 service nova] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:18:55.379682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:18:55.380358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:18:55.380636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:18:55.381023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:18:55.382969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:18:55.404997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:18:55.404997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4a0b-7d43-4a5b-b1d0-dfdd6041572f service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:18:55.418034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1172/2343] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:18:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:19:03.064388 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=65 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:03.502739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:03.504234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:03.529614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031580c-bd81-400d-baa8-0b1b14870828 service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:03.533347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:03.533964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:03.534244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:03.534519 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:03.536283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:03.615847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:03.617368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:03.630151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1172/2344] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:05.411568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=15 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:13.623972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=40 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:16.406485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:16.406895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:16.407035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:16.429387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] MFA Rules not processed for user `c6291932ceb84b03924c0f056f29c2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:16.430598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:16.438225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1173/2345] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:26.423201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:26.587106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:26.634007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f3648-f6c1-4a6d-8ffc-411430b78948 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:26.637971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:26.638800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:26.639074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:26.639345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:26.641227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:26.741295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:26.742736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:26.767566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1173/2346] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:30.242008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:30.243660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:30.318732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:30.323524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:30.339986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2a08-b333-43dc-916e-94083ae5480f service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:30.356655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:30.357696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:30.358506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:30.358966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] PATH_INFO: `/v3/users/97115d07acc5464bb7b097039471fee6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:30.361249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97115d07acc5464bb7b097039471fee6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:30.363661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:30.384305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14046c-d1a4-4b59-bbf3-1dfc46644d28 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:30.391858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:30.395465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:30.396087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:30.396609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:30.399589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:30.416503 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:30.420311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:30.430094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97115d07acc5464bb7b097039471fee6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.430816 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97115d07acc5464bb7b097039471fee6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.436256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1174/2347] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:30.452567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97115d07acc5464bb7b097039471fee6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.474827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97115d07acc5464bb7b097039471fee6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.485219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-92903f59-916c-450f-b786-a38fb70930d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97115d07acc5464bb7b097039471fee6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.495305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1174/2348] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:30 2022] DELETE /identity/v3/users/97115d07acc5464bb7b097039471fee6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:30.696461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:30.711274 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd77d1d-69b8-4508-9b72-26a33ed6d1b8 service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:30.773662 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:30.774315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:30.774609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:30.774866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] PATH_INFO: `/v3/projects/2b8b6c88af94417c840fdafd1fe431b7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:30.776023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8b6c88af94417c840fdafd1fe431b7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:30.779348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:30.827128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8b6c88af94417c840fdafd1fe431b7', 'request_id': 'req-f590dd53-b52d-4a6d-a988-9637fd9b88fe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.833326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8b6c88af94417c840fdafd1fe431b7', 'request_id': 'req-f590dd53-b52d-4a6d-a988-9637fd9b88fe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.840719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8b6c88af94417c840fdafd1fe431b7', 'request_id': 'req-f590dd53-b52d-4a6d-a988-9637fd9b88fe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.854203 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] The token cache is being invalidate because project 2b8b6c88af94417c840fdafd1fe431b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:30.854461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.855124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-f590dd53-b52d-4a6d-a988-9637fd9b88fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:30.856781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1175/2349] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:30 2022] DELETE /identity/v3/projects/2b8b6c88af94417c840fdafd1fe431b7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:30.862415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:30.862703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:30.862982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:30.876755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:30.928813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:30.930267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:30.998358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1175/2350] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:31.006239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.006653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.006772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.029591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:31.030841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:31.039251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1176/2351] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:31.044012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.054125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b5226-a00f-4785-8607-f592fbccfc10 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.059714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.060394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.060689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.060998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.061905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:19:31.062214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.063875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.077605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1176/2352] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:19:31.082440 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.091986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e8d73-8de5-438a-87e1-6501f758f490 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.096738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.098093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.098093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.098093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.098673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.100522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.113291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:31.143770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1177/2353] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:31 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:31.148120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.158750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bcd3e2-59cd-4ed2-9039-6b7746cd2109 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.224135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.225562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.225703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.225974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.226656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.228580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.256260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1177/2354] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:31 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:31.260304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.272709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563dc2f-cd74-4720-8819-65630f44057b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.278479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.279258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.279647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.280033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.281006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.283843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.296435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1178/2355] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:31.300645 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.312631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbc17e-1b32-4e6f-a206-4cfe69d878de admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.325185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305572ea-973b-43bf-8d3e-3ecb59b43cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.326112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305572ea-973b-43bf-8d3e-3ecb59b43cce admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.326594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305572ea-973b-43bf-8d3e-3ecb59b43cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.327069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305572ea-973b-43bf-8d3e-3ecb59b43cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.328439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305572ea-973b-43bf-8d3e-3ecb59b43cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.330509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305572ea-973b-43bf-8d3e-3ecb59b43cce admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.343050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1178/2356] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:31.347018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.358545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49d6aa-eca0-4c04-9eac-9b6c2fa4edb1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.363429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.364118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.364394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.364663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] PATH_INFO: `/v3/projects/e976ab0b7b8b4b638f1041dcf81c2a3f/users/ca259322061741368edbbf2f0e5618ca/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.382077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e976ab0b7b8b4b638f1041dcf81c2a3f, user_id=ca259322061741368edbbf2f0e5618ca, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.384750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.400705 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1179/2357] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:31 2022] PUT /identity/v3/projects/e976ab0b7b8b4b638f1041dcf81c2a3f/users/ca259322061741368edbbf2f0e5618ca/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:31.412996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.413280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.413560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.444215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] MFA Rules not processed for user `ca259322061741368edbbf2f0e5618ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:31.445483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:31.482227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:31.521261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1179/2358] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:31.525955 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.606043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f16b9e-bd09-49f6-8483-547f74b04a42 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.611064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.611702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.612016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.612293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.612983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.614927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.659195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1180/2359] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:31 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:31.664699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.678047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf0768-47b4-4ac6-adee-7118bbe61dd6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.741850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.742474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.742752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.743026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.743699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.745923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.768804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1180/2360] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:31 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:31.772939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.784749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a560f-2289-4429-8b57-57161f8eb052 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.789851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.790480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.790759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.791073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.792299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.794626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.805930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1181/2361] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:31.810451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.820537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3b50-0a1a-41b2-bf79-cc8bd546d239 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.825521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.826205 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.826483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.826757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] PATH_INFO: `/v3/projects/27511858a0114a56b1f45c56012071f8/users/878c6ac7c99d4a729153ab20b8c491d5/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.846207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27511858a0114a56b1f45c56012071f8, user_id=878c6ac7c99d4a729153ab20b8c491d5, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.850966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.886005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1181/2362] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:31 2022] PUT /identity/v3/projects/27511858a0114a56b1f45c56012071f8/users/878c6ac7c99d4a729153ab20b8c491d5/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:31.891892 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:31.903316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434ee9-9982-49f6-83cf-5888e76da004 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:31.973902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3001e5-3066-4f0f-8911-127f135a1de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:31.974530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3001e5-3066-4f0f-8911-127f135a1de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:31.974810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3001e5-3066-4f0f-8911-127f135a1de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:31.975083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3001e5-3066-4f0f-8911-127f135a1de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:31.975781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3001e5-3066-4f0f-8911-127f135a1de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:31.979601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3001e5-3066-4f0f-8911-127f135a1de7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:31.992675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1182/2363] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:19:31.998453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:32.017109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfb733-236b-43f1-949b-9b93ccc21f5c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:32.022980 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:32.023596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:32.024031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:32.024300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] PATH_INFO: `/v3/projects/27511858a0114a56b1f45c56012071f8/users/878c6ac7c99d4a729153ab20b8c491d5/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:32.026845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27511858a0114a56b1f45c56012071f8, user_id=878c6ac7c99d4a729153ab20b8c491d5, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:32.029745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:32.044779 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1182/2364] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:31 2022] PUT /identity/v3/projects/27511858a0114a56b1f45c56012071f8/users/878c6ac7c99d4a729153ab20b8c491d5/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:32.056268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:32.056555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:32.056949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:32.094862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] MFA Rules not processed for user `878c6ac7c99d4a729153ab20b8c491d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:32.096679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:32.102835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:32.196380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1183/2365] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:32.209172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:32.211507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:32.331732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a781d1-143f-49e4-8f8b-8bcdd21c9275 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:32.335625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdfc9aa-e389-4822-b529-d7de4f5bde0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:32.336257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc9aa-e389-4822-b529-d7de4f5bde0f service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:32.336539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc9aa-e389-4822-b529-d7de4f5bde0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:32.336818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfc9aa-e389-4822-b529-d7de4f5bde0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:32.347782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfc9aa-e389-4822-b529-d7de4f5bde0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:32.348952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdfc9aa-e389-4822-b529-d7de4f5bde0f service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:32.361462 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1183/2366] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:32.857894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:32.954944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0aef2-492e-4bec-a11a-4a5b738ee30a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:32.965035 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:32.965721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:32.966013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:32.966293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] PATH_INFO: `/v3/users/ca259322061741368edbbf2f0e5618ca` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:32.967554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca259322061741368edbbf2f0e5618ca)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:32.969821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:33.032256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca259322061741368edbbf2f0e5618ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.040279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca259322061741368edbbf2f0e5618ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.050177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca259322061741368edbbf2f0e5618ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.050922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca259322061741368edbbf2f0e5618ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.051562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca259322061741368edbbf2f0e5618ca'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.077435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1184/2367] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:32 2022] DELETE /identity/v3/users/ca259322061741368edbbf2f0e5618ca => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:33.082271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665237d-3022-479e-8504-8df18804774c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:33.082739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665237d-3022-479e-8504-8df18804774c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:33.083333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665237d-3022-479e-8504-8df18804774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:33.107338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9665237d-3022-479e-8504-8df18804774c None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:33.108706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9665237d-3022-479e-8504-8df18804774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:33.113373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:33.185786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1184/2368] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:33.196070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:33.197603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:33.244057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:33.307761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb209db-6af0-4ebe-aeb4-6df246dccb97 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:33.311599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:33.312237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:33.312534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:33.312782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:33.332997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:33.335392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:33.348514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1185/2369] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:33.675356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665237d-3022-479e-8504-8df18804774c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:33.677561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9665237d-3022-479e-8504-8df18804774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:33.706970 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665237d-3022-479e-8504-8df18804774c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:33.712063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:33.712737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:33.713047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:33.713327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] PATH_INFO: `/v3/projects/e976ab0b7b8b4b638f1041dcf81c2a3f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:33.715179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e976ab0b7b8b4b638f1041dcf81c2a3f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:33.717259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:33.784117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e976ab0b7b8b4b638f1041dcf81c2a3f', 'request_id': 'req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.794123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e976ab0b7b8b4b638f1041dcf81c2a3f', 'request_id': 'req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.801035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e976ab0b7b8b4b638f1041dcf81c2a3f', 'request_id': 'req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.815505 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] The token cache is being invalidate because project e976ab0b7b8b4b638f1041dcf81c2a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:33.815810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.816494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.818096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1185/2370] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:33 2022] DELETE /identity/v3/projects/e976ab0b7b8b4b638f1041dcf81c2a3f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:33.822151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:33.832535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e01bb-d2f2-4510-ad3b-1a8aec105df7 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:33.903828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:33.904551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:33.904756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:33.905072 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] PATH_INFO: `/v3/users/878c6ac7c99d4a729153ab20b8c491d5` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:33.906334 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=878c6ac7c99d4a729153ab20b8c491d5)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:33.908498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:33.967626 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878c6ac7c99d4a729153ab20b8c491d5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.975727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878c6ac7c99d4a729153ab20b8c491d5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.985758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878c6ac7c99d4a729153ab20b8c491d5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.986489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878c6ac7c99d4a729153ab20b8c491d5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:33.987140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-4ddc01ac-cf23-4b9f-b350-76da2dff2a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878c6ac7c99d4a729153ab20b8c491d5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:34.006206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1186/2371] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:33 2022] DELETE /identity/v3/users/878c6ac7c99d4a729153ab20b8c491d5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:34.347686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:34.352933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:34.366521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f80a2-1ebe-4e9b-a84a-4760a5cb2105 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:34.437502 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:34.438156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.438440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.438720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] PATH_INFO: `/v3/projects/27511858a0114a56b1f45c56012071f8` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.439902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=27511858a0114a56b1f45c56012071f8)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:34.441869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:34.482955 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:34.495164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27511858a0114a56b1f45c56012071f8', 'request_id': 'req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:34.509103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27511858a0114a56b1f45c56012071f8', 'request_id': 'req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:34.525344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27511858a0114a56b1f45c56012071f8', 'request_id': 'req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:34.546585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] The token cache is being invalidate because project 27511858a0114a56b1f45c56012071f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:34.547102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:34.548163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6d89545e-c113-4644-9d6d-cc57f0cb2c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:34.552996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1186/2372] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:34 2022] DELETE /identity/v3/projects/27511858a0114a56b1f45c56012071f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:34.556587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.556883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.557307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.583481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:34.584737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:34.624676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:34.671798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1187/2373] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:34.673303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.673596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.673873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.699266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:34.700517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:34.710433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1187/2374] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:19:34.714965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:34.727132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14127cc0-910f-4218-b816-dca1df5a44aa None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:34.732229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:34.733175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.733460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.733736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.734496 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:19:34.734802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:34.736473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:34.755324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1188/2375] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:19:34.756682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:34.771966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f418f4e-fafd-43d9-9327-a9fb3ac26b55 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:34.787820 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:34.788478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.788746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.789056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.789852 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:34.792243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:34.830797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1188/2376] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:34 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:34.833164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:34.844547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa828453-d8ae-4ab8-a751-e1904f31cfd6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:34.934015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:34.934754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.934941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.935288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.935960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:34.937933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:34.967269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1189/2377] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:34 2022] POST /identity/v3/users => generated 284 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:34.971716 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:34.983338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a33284-9697-4715-be37-1f05c647560c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:34.991908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:34.992576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:34.992860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:34.993171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:34.993881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:34.995593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.009013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1189/2378] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:35.013246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.023790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a6712-1957-474b-b7a0-b6196e2c0e80 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.029284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb4546-41e4-4852-bf65-bc0fc372010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.029918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb4546-41e4-4852-bf65-bc0fc372010b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.030195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb4546-41e4-4852-bf65-bc0fc372010b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.030464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb4546-41e4-4852-bf65-bc0fc372010b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.031360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb4546-41e4-4852-bf65-bc0fc372010b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.033644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb4546-41e4-4852-bf65-bc0fc372010b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.046061 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1190/2379] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:35.050116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.060483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925c16-6731-4c9f-bec6-7534b31fbf7f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.065387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.066266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.066449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.066706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] PATH_INFO: `/v3/projects/d6b9aa2081a74bcd8d5136e8398059ee/users/d8c4ca8ce6144267b13628385ce3d21f/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.083854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6b9aa2081a74bcd8d5136e8398059ee, user_id=d8c4ca8ce6144267b13628385ce3d21f, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.086569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.103247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1190/2380] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:35 2022] PUT /identity/v3/projects/d6b9aa2081a74bcd8d5136e8398059ee/users/d8c4ca8ce6144267b13628385ce3d21f/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:19:35.116542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.117059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.117341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.149765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] MFA Rules not processed for user `d8c4ca8ce6144267b13628385ce3d21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:35.151713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:35.230674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1191/2381] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:35.234734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.305586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a6bf6-e548-4079-b429-226f95023385 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.310421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.311045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.311324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.311600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.312280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.314239 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.344760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1191/2382] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:35 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:35.348654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.360497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5334388-93a3-49ed-93a5-8dc51bde5b3b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.388113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:35.421257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.421902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.422182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.422453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.423132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.425134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.452804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1192/2383] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:35 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:35.454349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.467449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493e30c-3a2c-4d72-9984-a03cf8872d0b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.472571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.473168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.473455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.473731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.474433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.476047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.487099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1192/2384] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:35.495366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.506901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc602492-d763-423f-9e51-da91a5dccb73 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.512704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.513423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.513704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.513981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] PATH_INFO: `/v3/projects/a60747ddaecf48e9a5ba62296d4336af/users/489103c29728403db972b962e22c40a0/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.530740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60747ddaecf48e9a5ba62296d4336af, user_id=489103c29728403db972b962e22c40a0, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.533535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.548627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1193/2385] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:35 2022] PUT /identity/v3/projects/a60747ddaecf48e9a5ba62296d4336af/users/489103c29728403db972b962e22c40a0/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:35.552597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.563786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b7889-4285-4fad-b0c8-8fc2e4790591 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.596397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:35.632456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79f590-9134-49a6-999b-eb6958c059ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.633119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79f590-9134-49a6-999b-eb6958c059ec admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.633398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79f590-9134-49a6-999b-eb6958c059ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.633671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79f590-9134-49a6-999b-eb6958c059ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.634423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79f590-9134-49a6-999b-eb6958c059ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.636850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79f590-9134-49a6-999b-eb6958c059ec admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.647881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1193/2386] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:35.652185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.663814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c651a4f-363f-42f6-8bff-b8763b3bbe7d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.670175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.670870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.671159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.671431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] PATH_INFO: `/v3/projects/a60747ddaecf48e9a5ba62296d4336af/users/489103c29728403db972b962e22c40a0/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.673482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60747ddaecf48e9a5ba62296d4336af, user_id=489103c29728403db972b962e22c40a0, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.676394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.688018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1194/2387] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:35 2022] PUT /identity/v3/projects/a60747ddaecf48e9a5ba62296d4336af/users/489103c29728403db972b962e22c40a0/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:35.699140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.699423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.699926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.728827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] MFA Rules not processed for user `489103c29728403db972b962e22c40a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:35.730106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:35.803348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1194/2388] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:35 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:35.815025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:35.816813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:35.882967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f2fc-c3b9-43d8-93ff-8bbe068c63b4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:35.886828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dbc9d-28ac-4e7d-8900-6f57c6eba261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:35.887461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dbc9d-28ac-4e7d-8900-6f57c6eba261 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:35.887743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dbc9d-28ac-4e7d-8900-6f57c6eba261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:35.888017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dbc9d-28ac-4e7d-8900-6f57c6eba261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:35.898073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dbc9d-28ac-4e7d-8900-6f57c6eba261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:35.899519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3dbc9d-28ac-4e7d-8900-6f57c6eba261 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:35.910907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1195/2389] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:35 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:19:37.305238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:37.371825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14eadf-02be-4716-a7d8-6d84ac0eed93 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:37.376881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:37.377558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:37.377831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:37.378105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] PATH_INFO: `/v3/users/d8c4ca8ce6144267b13628385ce3d21f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:37.379254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c4ca8ce6144267b13628385ce3d21f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:37.381212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:37.428663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c4ca8ce6144267b13628385ce3d21f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:37.429329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c4ca8ce6144267b13628385ce3d21f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:37.436323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:37.442855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c4ca8ce6144267b13628385ce3d21f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:37.451048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c4ca8ce6144267b13628385ce3d21f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:37.458144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c4ca8ce6144267b13628385ce3d21f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:37.464878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1195/2390] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:37 2022] DELETE /identity/v3/users/d8c4ca8ce6144267b13628385ce3d21f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:37.469527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:37.470010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:37.470135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:37.489086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:37.490275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:37.550472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1196/2391] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:37.560478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:37.562015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:37.619168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07df22a-3f3f-426c-80c3-b0443ef6a3dd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:37.623322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:37.623961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:37.624242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:37.624564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:37.633929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:37.635942 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:37.645531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1196/2392] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:37.922798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:37.924350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:37.938594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dc8e9-b643-459f-a968-c306cc5650c7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:37.943241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:37.943890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:37.944172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:37.944448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] PATH_INFO: `/v3/projects/d6b9aa2081a74bcd8d5136e8398059ee` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:37.945700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6b9aa2081a74bcd8d5136e8398059ee)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:37.948413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:37.972373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:38.006550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6b9aa2081a74bcd8d5136e8398059ee', 'request_id': 'req-c00a4ef5-bdcf-4cbb-821d-b4293343d990'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.013818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6b9aa2081a74bcd8d5136e8398059ee', 'request_id': 'req-c00a4ef5-bdcf-4cbb-821d-b4293343d990'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.019434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6b9aa2081a74bcd8d5136e8398059ee', 'request_id': 'req-c00a4ef5-bdcf-4cbb-821d-b4293343d990'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.032043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] The token cache is being invalidate because project d6b9aa2081a74bcd8d5136e8398059ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:38.032330 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.032998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.034520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1197/2393] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:37 2022] DELETE /identity/v3/projects/d6b9aa2081a74bcd8d5136e8398059ee => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:38.038574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:38.049290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f4188-9ce3-44bc-a0b4-f20de8e61967 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:38.110620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:38.111263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.111543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.111819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] PATH_INFO: `/v3/users/489103c29728403db972b962e22c40a0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.112996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=489103c29728403db972b962e22c40a0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:38.114909 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:38.162988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '489103c29728403db972b962e22c40a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.163633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '489103c29728403db972b962e22c40a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.175737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '489103c29728403db972b962e22c40a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.183496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '489103c29728403db972b962e22c40a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.189380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:38.192856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9007b182-dc4e-4cb7-a580-e7ff1a5c4138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '489103c29728403db972b962e22c40a0'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.204747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1197/2394] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:38 2022] DELETE /identity/v3/users/489103c29728403db972b962e22c40a0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:38.545050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:38.546612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:38.562063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a4ef5-bdcf-4cbb-821d-b4293343d990 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:38.622504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:38.623153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.623441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.623717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] PATH_INFO: `/v3/projects/a60747ddaecf48e9a5ba62296d4336af` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.625176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60747ddaecf48e9a5ba62296d4336af)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:38.627108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:38.674441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60747ddaecf48e9a5ba62296d4336af', 'request_id': 'req-5787e3f2-7d37-466a-9ce4-b894cf6da68f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.680561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60747ddaecf48e9a5ba62296d4336af', 'request_id': 'req-5787e3f2-7d37-466a-9ce4-b894cf6da68f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.686559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60747ddaecf48e9a5ba62296d4336af', 'request_id': 'req-5787e3f2-7d37-466a-9ce4-b894cf6da68f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.698813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] The token cache is being invalidate because project a60747ddaecf48e9a5ba62296d4336af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:38.699101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.699729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5787e3f2-7d37-466a-9ce4-b894cf6da68f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:38.701395 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1198/2395] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:38 2022] DELETE /identity/v3/projects/a60747ddaecf48e9a5ba62296d4336af => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:38.706451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.707013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.707013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.726824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:38.728019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:38.788232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1198/2396] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:38.796835 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.797158 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.797430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.816743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:38.817995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:38.826795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1199/2397] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:38.831101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:38.841625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389be52d-e2ce-4320-ab46-d381e080f8a2 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:38.846214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:38.846817 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.847097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.847366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.848060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:19:38.848362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:38.849981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:38.861924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1199/2398] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:19:38.866471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:38.876849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa0f47-86e5-4ac5-9abf-980551afb5b2 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:38.882320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:38.882990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.883304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.883611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.884447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:38.887192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:38.915655 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1200/2399] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:38 2022] POST /identity/v3/projects => generated 401 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:38.920330 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:38.930432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc3aaa-674f-4f94-b009-72ab07f59f2b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:38.987016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:38.987633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:38.987912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:38.988182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:38.988855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:38.990808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.013767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1200/2400] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:38 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:39.017966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.029201 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b1fc8-7ee6-4ce5-9010-3220bd81de4c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.033805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.034546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.034826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.035094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.035782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.037442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.039973 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:39.050534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1201/2401] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:39.054611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.064704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba7c28-7881-4fec-85b3-9fba94587680 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.069381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21070fc9-b726-4667-ad5f-4bcc4393daeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.070100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21070fc9-b726-4667-ad5f-4bcc4393daeb admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.070387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21070fc9-b726-4667-ad5f-4bcc4393daeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.070662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21070fc9-b726-4667-ad5f-4bcc4393daeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.071353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21070fc9-b726-4667-ad5f-4bcc4393daeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.073573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21070fc9-b726-4667-ad5f-4bcc4393daeb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.085093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1201/2402] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:39.090181 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.100353 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17753a-fd8e-4cef-8a2a-e265c91f0cdd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.105069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.105725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.106009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.106283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] PATH_INFO: `/v3/projects/4b961dd28bc3427aa1272e3192182070/users/1ac217e3f5d84df69b7453a90f29f51f/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.121238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b961dd28bc3427aa1272e3192182070, user_id=1ac217e3f5d84df69b7453a90f29f51f, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.123919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.138745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1202/2403] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:39 2022] PUT /identity/v3/projects/4b961dd28bc3427aa1272e3192182070/users/1ac217e3f5d84df69b7453a90f29f51f/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:39.150233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.150523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.151421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.179394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] MFA Rules not processed for user `1ac217e3f5d84df69b7453a90f29f51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:39.180703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:39.230967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:39.250048 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1202/2404] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:39.255210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.321346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c2fc1-25f2-4db2-a7a5-7ebea7926c58 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.326836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.327464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.327741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.328007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.328761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.330701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.359237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1203/2405] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:39 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:39.362717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.374209 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694cc0-f185-4fad-9cff-9d0571e80224 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.433952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.434482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.434758 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.435030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.435719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.437689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.459986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1203/2406] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:39 2022] POST /identity/v3/users => generated 307 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:39.464240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.474752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7263-d382-4e61-900e-1119c2404b44 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.479847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.480465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.480748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.481128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.481822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.483445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.495434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1204/2407] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:39.500082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.512375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbf6ab-6760-4bb5-b1b1-b25899ce944d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.517708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.518423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.518754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.519045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] PATH_INFO: `/v3/projects/d1ad4bb2ffa54f479469757c97c46dcd/users/f7ea272dc23f4015803acbf4821b27cc/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.538792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ad4bb2ffa54f479469757c97c46dcd, user_id=f7ea272dc23f4015803acbf4821b27cc, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.542517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.556604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1204/2408] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:39 2022] PUT /identity/v3/projects/d1ad4bb2ffa54f479469757c97c46dcd/users/f7ea272dc23f4015803acbf4821b27cc/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:39.561778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.572616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754a7bf-42b6-430b-8bca-9977122f5284 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.639235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e18140-82a9-4d3e-8db1-a53efc825c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.639862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e18140-82a9-4d3e-8db1-a53efc825c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.640139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e18140-82a9-4d3e-8db1-a53efc825c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.640407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e18140-82a9-4d3e-8db1-a53efc825c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.641159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e18140-82a9-4d3e-8db1-a53efc825c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.642770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e18140-82a9-4d3e-8db1-a53efc825c84 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.653676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1205/2409] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:19:39.658083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.670209 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fb79d-f4da-4c76-b8b9-c2a567dbbb46 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.676344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.677244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.677527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.677799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] PATH_INFO: `/v3/projects/d1ad4bb2ffa54f479469757c97c46dcd/users/f7ea272dc23f4015803acbf4821b27cc/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.679886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ad4bb2ffa54f479469757c97c46dcd, user_id=f7ea272dc23f4015803acbf4821b27cc, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.684095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.695438 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1205/2410] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:39 2022] PUT /identity/v3/projects/d1ad4bb2ffa54f479469757c97c46dcd/users/f7ea272dc23f4015803acbf4821b27cc/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:19:39.705622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.705883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.706148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.734935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] MFA Rules not processed for user `f7ea272dc23f4015803acbf4821b27cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:39.736137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:39.768541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:39.808293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1206/2411] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:39 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:39.820820 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:39.822726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:39.881677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49348a53-0ac7-4abc-bbe4-b2890b64aa27 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:39.885404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:39.886015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:39.886298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:39.886579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:39.895797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:39.897368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:39.907440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1206/2412] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:39 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:39.991755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:40.002358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57ecc6-0c63-47bc-98a8-f3e351ad1f79 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:40.006113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9a583-55c7-495b-bfdc-95b1a3e29efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:40.006731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9a583-55c7-495b-bfdc-95b1a3e29efa service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:40.007013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9a583-55c7-495b-bfdc-95b1a3e29efa service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:40.007288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9a583-55c7-495b-bfdc-95b1a3e29efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:40.073100 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9a583-55c7-495b-bfdc-95b1a3e29efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:40.074784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9a583-55c7-495b-bfdc-95b1a3e29efa service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:40.085896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1207/2413] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:44.349448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:44.410329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:44.429279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a239b-9606-470e-a399-f227c59629ec service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:44.434589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:44.435242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:44.435523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:44.435799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] PATH_INFO: `/v3/users/1ac217e3f5d84df69b7453a90f29f51f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:44.437030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac217e3f5d84df69b7453a90f29f51f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:44.438966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:44.488283 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac217e3f5d84df69b7453a90f29f51f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:44.488988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac217e3f5d84df69b7453a90f29f51f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:44.502340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac217e3f5d84df69b7453a90f29f51f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:44.512289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac217e3f5d84df69b7453a90f29f51f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:44.520398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac217e3f5d84df69b7453a90f29f51f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:44.526764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1207/2414] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:44 2022] DELETE /identity/v3/users/1ac217e3f5d84df69b7453a90f29f51f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:19:44.531953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:44.532763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:44.532763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:44.555246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:44.556449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:44.622015 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1208/2415] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:44.636551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:44.638203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:44.701708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24041af0-6746-42df-9899-9ee95211f652 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:44.706913 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:44.707700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:44.708109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:44.708390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:44.721834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:44.723827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:44.737883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1208/2416] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:19:45.033919 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:45.035055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:45.048990 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eff84b-7e42-4adb-af43-b9a8c2254d87 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:45.054450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:45.055704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:45.056242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:45.056691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] PATH_INFO: `/v3/projects/4b961dd28bc3427aa1272e3192182070` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:45.059695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b961dd28bc3427aa1272e3192182070)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:45.062408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:45.117239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b961dd28bc3427aa1272e3192182070', 'request_id': 'req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.127283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b961dd28bc3427aa1272e3192182070', 'request_id': 'req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.134737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b961dd28bc3427aa1272e3192182070', 'request_id': 'req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.139084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:45.158082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] The token cache is being invalidate because project 4b961dd28bc3427aa1272e3192182070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:45.158550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.159868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.161741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1209/2417] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:45 2022] DELETE /identity/v3/projects/4b961dd28bc3427aa1272e3192182070 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:45.165879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:45.177118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7630d-49a4-4d13-ba81-25579e46a149 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:45.253989 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:45.254650 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:45.254915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:45.255338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] PATH_INFO: `/v3/users/f7ea272dc23f4015803acbf4821b27cc` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:45.256555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ea272dc23f4015803acbf4821b27cc)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:45.258536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:45.271144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:45.314278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ea272dc23f4015803acbf4821b27cc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.314967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ea272dc23f4015803acbf4821b27cc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.332037 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ea272dc23f4015803acbf4821b27cc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.340899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ea272dc23f4015803acbf4821b27cc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.349869 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-94ff4f0b-467e-4cc5-906b-f5352b30cccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ea272dc23f4015803acbf4821b27cc'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.360512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1209/2418] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:45 2022] DELETE /identity/v3/users/f7ea272dc23f4015803acbf4821b27cc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:45.686127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:45.688649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:45.706380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6ea4-abb5-4fe9-9301-d8cc0d884ffe admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:45.770721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:45.771672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:45.772298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:45.772595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] PATH_INFO: `/v3/projects/d1ad4bb2ffa54f479469757c97c46dcd` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:45.773827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ad4bb2ffa54f479469757c97c46dcd)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:45.775754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:45.828867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ad4bb2ffa54f479469757c97c46dcd', 'request_id': 'req-6e114206-727b-4585-b1ca-8a360ffbc53d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.835528 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ad4bb2ffa54f479469757c97c46dcd', 'request_id': 'req-6e114206-727b-4585-b1ca-8a360ffbc53d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.841947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ad4bb2ffa54f479469757c97c46dcd', 'request_id': 'req-6e114206-727b-4585-b1ca-8a360ffbc53d'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.856633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] The token cache is being invalidate because project d1ad4bb2ffa54f479469757c97c46dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:45.856917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.857624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6e114206-727b-4585-b1ca-8a360ffbc53d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:45.859157 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1210/2419] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:45 2022] DELETE /identity/v3/projects/d1ad4bb2ffa54f479469757c97c46dcd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:45.864142 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:45.864435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:45.864718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:45.887575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:45.888814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:45.952242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1210/2420] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:45.960939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:45.961255 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:45.961538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:45.983974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:45.985227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:45.993569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1211/2421] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:45.998278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.008937 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452f10-f227-4889-949e-84c26d9c3740 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.014822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.015475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.015757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.016032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.016741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:19:46.017216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.019043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.032912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1211/2422] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:19:46.037346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.053468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d462c42-2b08-4822-9556-838f89ff5ad5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.058952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.059618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.059901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.060174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.060849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.063001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.092947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1212/2423] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:46 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:46.097349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.109181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38acb7-2312-4250-8b52-e90f2201ed35 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.171497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.172113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.172397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.172666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.173387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.175366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.197747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1212/2424] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:46 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:46.204423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.221933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2fd90-f3f7-43b5-9543-d2eb4cbbd0a4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.226644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.227275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.227551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.227826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.228520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.231109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.242968 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1213/2425] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:46.247746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.255925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:46.260465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfb3d1-e4b7-48e7-a8bb-50b3d9f5d199 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.269105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbf8ae-e63c-491c-b69f-bddae4cdd610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.269845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf8ae-e63c-491c-b69f-bddae4cdd610 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.270339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf8ae-e63c-491c-b69f-bddae4cdd610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.270968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf8ae-e63c-491c-b69f-bddae4cdd610 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.271681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbf8ae-e63c-491c-b69f-bddae4cdd610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.273611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbf8ae-e63c-491c-b69f-bddae4cdd610 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.289899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1213/2426] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:19:46.290875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.303041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1a971-8a2a-4e90-b4d4-84e3c40981d2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.308129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.309135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.309418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.309697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] PATH_INFO: `/v3/projects/21148d8178194d6c971f2d16a4d9d5b9/users/288a181f22634dfca6f7d229688ea77a/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.327166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21148d8178194d6c971f2d16a4d9d5b9, user_id=288a181f22634dfca6f7d229688ea77a, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.329983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.347065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1214/2427] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:46 2022] PUT /identity/v3/projects/21148d8178194d6c971f2d16a4d9d5b9/users/288a181f22634dfca6f7d229688ea77a/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:19:46.358020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.358829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.358829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.390708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] MFA Rules not processed for user `288a181f22634dfca6f7d229688ea77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:46.392145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:46.468625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1214/2428] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:46.473838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.498464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:46.560636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e246609-a667-4764-89ec-c9f777fad9e6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.565912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.566547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.566827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.567101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.567768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.569738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.602947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1215/2429] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:46 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:46.607964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.621746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004902e-158d-4d4c-aacd-a0411d67963f None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.695085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.695952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.696630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.697506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.699305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.701463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.732335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1215/2430] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:46 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:46.733690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.751091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea66c1-a89e-407c-b853-b8914bc91e3c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.757083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.757750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.758074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.758439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.759532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.762754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.778765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1216/2431] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:46.780211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.791070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8120f7-6384-4be9-bfd9-f9cefd7d3b32 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.797127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.797868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.798097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.798399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] PATH_INFO: `/v3/projects/6ef602f4a4764313bb7f74fd0c9531ae/users/dc612cdde4bf4c54bced5f00d1bfa5b9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.818092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ef602f4a4764313bb7f74fd0c9531ae, user_id=dc612cdde4bf4c54bced5f00d1bfa5b9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.820881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.836549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1216/2432] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:46 2022] PUT /identity/v3/projects/6ef602f4a4764313bb7f74fd0c9531ae/users/dc612cdde4bf4c54bced5f00d1bfa5b9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:46.840833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.851324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564af336-bd1b-4d81-998c-de730e7f83d4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.910290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314c573-c2fa-4f65-80a1-a970278bbf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.910824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314c573-c2fa-4f65-80a1-a970278bbf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.911104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314c573-c2fa-4f65-80a1-a970278bbf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.911377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314c573-c2fa-4f65-80a1-a970278bbf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.912086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314c573-c2fa-4f65-80a1-a970278bbf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.913745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314c573-c2fa-4f65-80a1-a970278bbf7b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.924341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1217/2433] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:46.928867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:46.941011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb62ba0-7fe9-4e56-b1e2-29f6a5b25922 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:46.945717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:46.946390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.946672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.946947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] PATH_INFO: `/v3/projects/6ef602f4a4764313bb7f74fd0c9531ae/users/dc612cdde4bf4c54bced5f00d1bfa5b9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:46.950011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ef602f4a4764313bb7f74fd0c9531ae, user_id=dc612cdde4bf4c54bced5f00d1bfa5b9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:46.952673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:46.963604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1217/2434] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:46 2022] PUT /identity/v3/projects/6ef602f4a4764313bb7f74fd0c9531ae/users/dc612cdde4bf4c54bced5f00d1bfa5b9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:46.974437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:46.974717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:46.974988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:47.004952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] MFA Rules not processed for user `dc612cdde4bf4c54bced5f00d1bfa5b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:47.006253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:47.081475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1218/2435] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:47.093275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:47.095105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:47.145663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:47.165449 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96863f76-466b-4c76-8273-9c8e9a248602 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:47.169827 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b84bd-93d6-433b-ad08-cc02d7cee566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:47.170450 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b84bd-93d6-433b-ad08-cc02d7cee566 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:47.170729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b84bd-93d6-433b-ad08-cc02d7cee566 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:47.170996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b84bd-93d6-433b-ad08-cc02d7cee566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:47.180987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b84bd-93d6-433b-ad08-cc02d7cee566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:47.182433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b84bd-93d6-433b-ad08-cc02d7cee566 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:47.193823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1218/2436] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:47.297518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:47.308819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311effd-d5a7-4236-a467-87629270ee43 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:47.312838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:47.313568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:47.313826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:47.314068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:47.329018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:47.386647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:47.388081 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:47.397484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1219/2437] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:50.087586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:50.088115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:50.089651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:50.122379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:50.123626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:50.188138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1219/2438] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:19:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:50.247739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:50.250018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:50.346030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696aa60-77e2-4ed2-b20f-087d0828dbc3 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:50.351981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745df38-8b91-42fa-8fd9-28f9ea526a94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:50.352810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745df38-8b91-42fa-8fd9-28f9ea526a94 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:50.353128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745df38-8b91-42fa-8fd9-28f9ea526a94 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:50.353379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745df38-8b91-42fa-8fd9-28f9ea526a94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:50.364007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745df38-8b91-42fa-8fd9-28f9ea526a94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:50.365760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745df38-8b91-42fa-8fd9-28f9ea526a94 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:50.378111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1220/2439] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:19:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:19:52.465908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:52.550117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3408ad0-023b-4bc3-a10e-ebb167405bc8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:52.557098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:52.558182 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:52.558466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:52.558824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] PATH_INFO: `/v3/users/288a181f22634dfca6f7d229688ea77a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:52.560314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288a181f22634dfca6f7d229688ea77a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:52.562362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:52.637806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288a181f22634dfca6f7d229688ea77a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:52.641789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288a181f22634dfca6f7d229688ea77a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:52.657210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288a181f22634dfca6f7d229688ea77a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:52.671406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288a181f22634dfca6f7d229688ea77a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:52.677677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:52.683871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288a181f22634dfca6f7d229688ea77a'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:52.692669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1220/2440] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:52 2022] DELETE /identity/v3/users/288a181f22634dfca6f7d229688ea77a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:52.697542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41290455-e903-4d48-9466-a210eff685d7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:52.698073 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41290455-e903-4d48-9466-a210eff685d7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:52.698729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41290455-e903-4d48-9466-a210eff685d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:52.725487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-41290455-e903-4d48-9466-a210eff685d7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:52.727133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-41290455-e903-4d48-9466-a210eff685d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:52.810941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1221/2441] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:19:52.818071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:52.821203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:52.928000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38bf9a-e6cb-44e8-87b7-d1a41f5484b2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:52.933633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:52.934679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:52.935068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:52.935446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:52.946908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:52.949781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:52.963284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1221/2442] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:53.394257 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41290455-e903-4d48-9466-a210eff685d7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:53.396951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-41290455-e903-4d48-9466-a210eff685d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:53.423215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41290455-e903-4d48-9466-a210eff685d7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:53.464234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:53.465178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:53.465573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:53.465905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] PATH_INFO: `/v3/projects/21148d8178194d6c971f2d16a4d9d5b9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:53.467740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=21148d8178194d6c971f2d16a4d9d5b9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:53.470288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:53.518587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:53.534171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21148d8178194d6c971f2d16a4d9d5b9', 'request_id': 'req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.542639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21148d8178194d6c971f2d16a4d9d5b9', 'request_id': 'req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.549268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21148d8178194d6c971f2d16a4d9d5b9', 'request_id': 'req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.577615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] The token cache is being invalidate because project 21148d8178194d6c971f2d16a4d9d5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:53.577906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.578535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.580118 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1222/2443] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:53 2022] DELETE /identity/v3/projects/21148d8178194d6c971f2d16a4d9d5b9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:19:53.584172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:53.596015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12f643-9650-4453-9b77-fbf403d8cbc0 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:53.676603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:53.677287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:53.677569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:53.677984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] PATH_INFO: `/v3/users/dc612cdde4bf4c54bced5f00d1bfa5b9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:53.679211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc612cdde4bf4c54bced5f00d1bfa5b9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:53.681216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:53.742262 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc612cdde4bf4c54bced5f00d1bfa5b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.743710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc612cdde4bf4c54bced5f00d1bfa5b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.758837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc612cdde4bf4c54bced5f00d1bfa5b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.770792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc612cdde4bf4c54bced5f00d1bfa5b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.781838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ce26d5c7-caa1-400b-b392-9112b2a1038b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc612cdde4bf4c54bced5f00d1bfa5b9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:53.790752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1222/2444] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:53 2022] DELETE /identity/v3/users/dc612cdde4bf4c54bced5f00d1bfa5b9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:19:54.191264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:54.193688 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:54.218130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6291dc7-b4d5-4cb8-b6be-5622d31aab8f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:54.305596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:54.306179 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.308515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.308515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] PATH_INFO: `/v3/projects/6ef602f4a4764313bb7f74fd0c9531ae` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.309956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ef602f4a4764313bb7f74fd0c9531ae)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:54.312017 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:54.390855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ef602f4a4764313bb7f74fd0c9531ae', 'request_id': 'req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:54.399511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ef602f4a4764313bb7f74fd0c9531ae', 'request_id': 'req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:54.410746 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ef602f4a4764313bb7f74fd0c9531ae', 'request_id': 'req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:54.428199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] The token cache is being invalidate because project 6ef602f4a4764313bb7f74fd0c9531ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:54.428611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:54.431455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-82f9eebe-a052-4499-977b-b9bd2c9e1c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:54.433446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1223/2445] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:54 2022] DELETE /identity/v3/projects/6ef602f4a4764313bb7f74fd0c9531ae => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:54.438416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.438815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.439297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.464624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:54.465968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:54.480639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:54.562317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1223/2446] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:54.563862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.564151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.564448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.592403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:54.593678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:54.605346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1224/2447] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:54.607191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:54.623532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756283d-bf16-4ade-9ec3-735bb4da9c97 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:54.629295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:54.629921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.630197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.630466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.631175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:19:54.631474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:54.633106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:54.648871 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1224/2448] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:19:54.654102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:54.667242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969af8e-4bf5-44a6-bd06-1bc62fed0042 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:54.673950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:54.675144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.675530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.675898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.676672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:54.679470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:54.713874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1225/2449] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:54 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:54.718517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:54.731726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2219a-cd6b-4514-9dc8-3dedac199bf4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:54.846489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:54.848162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.849237 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.849856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.851085 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:54.857968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:54.888856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1225/2450] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:54 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:19:54.894221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:54.926414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd513c8-baf8-4cb2-83a9-6801e3730894 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:54.936336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:54.937468 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:54.937613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:54.937846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:54.938550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:54.940483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:54.959027 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1226/2451] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:54.959740 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:54.986990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc231eaa-7b8b-4049-8df1-cbb3f4a352df admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:54.999677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff3e4c-50a9-4f09-95d9-7f6686b4986b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:55.002153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3e4c-50a9-4f09-95d9-7f6686b4986b admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.002153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3e4c-50a9-4f09-95d9-7f6686b4986b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.002153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff3e4c-50a9-4f09-95d9-7f6686b4986b admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.020514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3e4c-50a9-4f09-95d9-7f6686b4986b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:55.020514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff3e4c-50a9-4f09-95d9-7f6686b4986b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:55.029199 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1226/2452] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:55.031627 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:55.045547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945848f-1810-4618-9eaa-5e4a1e2793b9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:55.050667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:55.051449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.051778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.052056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] PATH_INFO: `/v3/projects/d5ab155c3d454f6091337681b48a3d58/users/74a5834d14924fb6ac7e23c08c22700d/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.066875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:55.074426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ab155c3d454f6091337681b48a3d58, user_id=74a5834d14924fb6ac7e23c08c22700d, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:55.077657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:55.096196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1227/2453] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:55 2022] PUT /identity/v3/projects/d5ab155c3d454f6091337681b48a3d58/users/74a5834d14924fb6ac7e23c08c22700d/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:55.108780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.109247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.109684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.163198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] MFA Rules not processed for user `74a5834d14924fb6ac7e23c08c22700d`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:55.164863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:55.270154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1227/2454] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:55.284460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:55.289604 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:55.391509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c40d42-6ea9-4e2e-a4fd-9403bab801ba admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:55.402473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332797-0b44-424e-9027-46c61f6fd916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:55.403266 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332797-0b44-424e-9027-46c61f6fd916 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.403620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332797-0b44-424e-9027-46c61f6fd916 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.404009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332797-0b44-424e-9027-46c61f6fd916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.415416 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6332797-0b44-424e-9027-46c61f6fd916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:55.417254 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6332797-0b44-424e-9027-46c61f6fd916 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:55.429883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1228/2455] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:55.483695 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:55.509749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:55.593724 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea7b8e-2894-4682-8c36-7b4a55a11f10 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:55.603705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:55.603705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.603705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.603705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] PATH_INFO: `/v3/users/74a5834d14924fb6ac7e23c08c22700d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.605704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a5834d14924fb6ac7e23c08c22700d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:55.608172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:55.677321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a5834d14924fb6ac7e23c08c22700d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:55.678074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a5834d14924fb6ac7e23c08c22700d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:55.696127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a5834d14924fb6ac7e23c08c22700d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:55.709126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a5834d14924fb6ac7e23c08c22700d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:55.725488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a5834d14924fb6ac7e23c08c22700d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:55.735781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1228/2456] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:19:55 2022] DELETE /identity/v3/users/74a5834d14924fb6ac7e23c08c22700d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:55.741494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.742303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.742303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.766639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:55.768692 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:55.840186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1229/2457] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:55.850116 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:55.851560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:55.927747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35fb9a5-2aa5-4fdc-b3c8-722873f23a2e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:55.932166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29302d3d-d65f-435d-98ae-ce341d6f8e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:55.932836 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29302d3d-d65f-435d-98ae-ce341d6f8e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:55.933147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29302d3d-d65f-435d-98ae-ce341d6f8e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:55.933423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29302d3d-d65f-435d-98ae-ce341d6f8e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:55.946274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29302d3d-d65f-435d-98ae-ce341d6f8e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:55.947950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29302d3d-d65f-435d-98ae-ce341d6f8e36 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:55.962106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1229/2458] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:19:56.325077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:56.327878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:56.355602 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4f43d-2b18-4298-9766-01a65d17dced None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:56.369102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:56.370660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:56.371133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:56.371419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] PATH_INFO: `/v3/projects/d5ab155c3d454f6091337681b48a3d58` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:56.372809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ab155c3d454f6091337681b48a3d58)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:56.375137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:56.454314 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ab155c3d454f6091337681b48a3d58', 'request_id': 'req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:56.461962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ab155c3d454f6091337681b48a3d58', 'request_id': 'req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:56.469422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ab155c3d454f6091337681b48a3d58', 'request_id': 'req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:56.485979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] The token cache is being invalidate because project d5ab155c3d454f6091337681b48a3d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:19:56.486521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:56.487408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-625d5d4b-ba1e-4a0d-8c25-4d34b3595843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:19:56.489570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1230/2459] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:19:56 2022] DELETE /identity/v3/projects/d5ab155c3d454f6091337681b48a3d58 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:56.496794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:56.497377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:56.497717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:56.522641 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:56.524004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:56.567833 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:56.621847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1230/2460] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:19:56.627378 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:56.628151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:56.628500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:56.681742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:56.683809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:56.700824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:56.703392 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1231/2461] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:56.718781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece33c5d-ecf3-4181-a634-951e8a400ce7 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:56.727517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:56.728264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:56.728614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:56.729503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:56.730351 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:19:56.730733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:56.732770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:56.751576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1231/2462] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:19:56.756243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:56.769064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b1f35-4afe-4c7a-ab1a-65731f44f686 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:56.783347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:56.783998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:56.784278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:56.784548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:56.785245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:56.787749 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:56.809111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:19:56.843435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1232/2463] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:19:56 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:56.849839 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:56.861323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4592c-874e-4fe4-90d5-70fd262faaf9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:56.941070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:56.941694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:56.942016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:56.942296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:56.943010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:56.945220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:56.975235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1232/2464] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:19:56 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:19:56.981365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:56.992695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ac70-ac0f-470f-ae4c-06836a14f59c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:56.999317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:57.000250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:57.000538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:57.000814 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:57.001546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:57.003175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:57.017931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1233/2465] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:19:57.019524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:57.032901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c54bd-77b8-475c-b373-40698e7faac2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:57.037820 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1fc373-454f-46a7-a138-aa8290a47d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:57.038451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fc373-454f-46a7-a138-aa8290a47d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:57.038736 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fc373-454f-46a7-a138-aa8290a47d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:57.039015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1fc373-454f-46a7-a138-aa8290a47d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:57.039719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fc373-454f-46a7-a138-aa8290a47d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:57.041368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1fc373-454f-46a7-a138-aa8290a47d25 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:57.055686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1233/2466] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:19:57.060704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:57.073034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0380b8-8f8a-4c83-8737-f3bac0b41f91 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:57.079781 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:57.080478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:57.080761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:57.081068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] PATH_INFO: `/v3/projects/8998c18e398049daa703fd694f5201b0/users/0f4d4413a3504604b8e76526b82c442b/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:57.109248 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8998c18e398049daa703fd694f5201b0, user_id=0f4d4413a3504604b8e76526b82c442b, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:57.112226 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:57.129104 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1234/2467] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:19:57 2022] PUT /identity/v3/projects/8998c18e398049daa703fd694f5201b0/users/0f4d4413a3504604b8e76526b82c442b/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:19:57.139753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036e9ce-04a6-49e3-96a4-7338b883d0e6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:57.140410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036e9ce-04a6-49e3-96a4-7338b883d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:57.140715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036e9ce-04a6-49e3-96a4-7338b883d0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:57.180282 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7036e9ce-04a6-49e3-96a4-7338b883d0e6 None None] MFA Rules not processed for user `0f4d4413a3504604b8e76526b82c442b`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:19:57.181562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7036e9ce-04a6-49e3-96a4-7338b883d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:57.271422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1234/2468] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:19:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:19:57.283736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:19:57.285609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:19:57.373987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613469c-25f1-4d51-b76d-2848f8d70024 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:19:57.378685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:19:57.379404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:19:57.379762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:19:57.384086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:19:57.395888 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:19:57.397933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:19:57.414523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1235/2469] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:19:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:07.268003 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:07.406833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:09.476275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:09.476829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:09.476829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:09.500221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:09.501504 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:09.583707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1235/2470] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:20:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:09.600405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:09.601929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:09.678915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e6aa3-fab1-4c4d-91b4-e70551ac7a41 service glance] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:09.683821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be21e1d-424c-42ed-b997-8188f77f2975 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:09.684443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be21e1d-424c-42ed-b997-8188f77f2975 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:09.684716 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be21e1d-424c-42ed-b997-8188f77f2975 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:09.685020 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be21e1d-424c-42ed-b997-8188f77f2975 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:09.693006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be21e1d-424c-42ed-b997-8188f77f2975 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:09.694595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be21e1d-424c-42ed-b997-8188f77f2975 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:09.703405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1236/2471] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:20:15.780057 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:15.903657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655623ff-a40d-4667-8f10-3871831d3766 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:15.926690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:15.933883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:15.934358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:15.934649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] PATH_INFO: `/v3/users/0f4d4413a3504604b8e76526b82c442b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:15.936105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4d4413a3504604b8e76526b82c442b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:15.938254 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:16.006960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4d4413a3504604b8e76526b82c442b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.007814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4d4413a3504604b8e76526b82c442b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.031601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4d4413a3504604b8e76526b82c442b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.041687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4d4413a3504604b8e76526b82c442b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.050668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4d4413a3504604b8e76526b82c442b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.063509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1236/2472] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:15 2022] DELETE /identity/v3/users/0f4d4413a3504604b8e76526b82c442b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:16.064907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26329-a818-4754-b366-631bc50eeeef None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:16.065464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26329-a818-4754-b366-631bc50eeeef None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:16.065571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df26329-a818-4754-b366-631bc50eeeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:16.085404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5df26329-a818-4754-b366-631bc50eeeef None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:16.086936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5df26329-a818-4754-b366-631bc50eeeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:16.152854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1237/2473] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:16.162745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:16.164251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:16.199160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:16.234470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be42317-a776-4dcf-acc1-24209e0c2e38 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:16.238576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962da08-d997-4074-8cc5-2bff8396ded0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:16.239271 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962da08-d997-4074-8cc5-2bff8396ded0 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:16.239615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962da08-d997-4074-8cc5-2bff8396ded0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:16.239963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962da08-d997-4074-8cc5-2bff8396ded0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:16.250849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962da08-d997-4074-8cc5-2bff8396ded0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:16.252397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962da08-d997-4074-8cc5-2bff8396ded0 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:16.265523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1237/2474] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:16.590939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26329-a818-4754-b366-631bc50eeeef None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:16.592985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5df26329-a818-4754-b366-631bc50eeeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:16.620486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df26329-a818-4754-b366-631bc50eeeef None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:16.640686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:16.641394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:16.641679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:16.641956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] PATH_INFO: `/v3/projects/8998c18e398049daa703fd694f5201b0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:16.643185 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8998c18e398049daa703fd694f5201b0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:16.645523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:16.742371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8998c18e398049daa703fd694f5201b0', 'request_id': 'req-bd9d7f33-5c12-4a3d-b817-14d494f4084a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.756912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8998c18e398049daa703fd694f5201b0', 'request_id': 'req-bd9d7f33-5c12-4a3d-b817-14d494f4084a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.767234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8998c18e398049daa703fd694f5201b0', 'request_id': 'req-bd9d7f33-5c12-4a3d-b817-14d494f4084a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.793370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] The token cache is being invalidate because project 8998c18e398049daa703fd694f5201b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:16.795368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.796163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bd9d7f33-5c12-4a3d-b817-14d494f4084a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:16.798198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1238/2475] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:16 2022] DELETE /identity/v3/projects/8998c18e398049daa703fd694f5201b0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:16.802861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:16.803454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:16.803454 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:16.825494 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:16.826727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:16.935446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1238/2476] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:16.946237 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:16.947777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:16.948275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:16.975573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:16.976935 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:16.986519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1239/2477] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:16.990891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.007674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8875b14-8865-4992-9bbb-73b4e11df71a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.036380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.037051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.037332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.037612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.038318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:17.038624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.040317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.067845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1239/2478] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:20:17.074088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.088460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4585a8-5fec-44d4-8fcb-a7d753b5e63a None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.102569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.103855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.104198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.104568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.105343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.115162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.123861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:17.193328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1240/2479] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:17 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:17.196614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.211558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758c74b-4a2a-4785-a097-b0a344da5a38 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.306944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.307656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.307977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.308370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.309577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.311531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.338009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1240/2480] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:17 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:17.342415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.354075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ee810-0d42-4939-8699-3d732c05bf4f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.365240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.366090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.366427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.366808 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.367676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.370890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.389301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1241/2481] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:17.393032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.404912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bff09b-df30-4959-95d3-1cf8e7fbefe0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.410367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bac186-5a5e-473e-98de-b474c81af6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.411042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac186-5a5e-473e-98de-b474c81af6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.411381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac186-5a5e-473e-98de-b474c81af6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.411660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bac186-5a5e-473e-98de-b474c81af6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.412385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bac186-5a5e-473e-98de-b474c81af6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.414034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bac186-5a5e-473e-98de-b474c81af6fb admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.428709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1241/2482] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:17.434998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.445886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9b3c3-9458-4e0e-8c2b-f9c004d5f566 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.451547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.452830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.453310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.453765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] PATH_INFO: `/v3/projects/011891ff0c69442eafd640a97a958552/users/182efff7efe74024b8ec15d50b6e5f2c/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.472340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=011891ff0c69442eafd640a97a958552, user_id=182efff7efe74024b8ec15d50b6e5f2c, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.474989 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.489912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1242/2483] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:17 2022] PUT /identity/v3/projects/011891ff0c69442eafd640a97a958552/users/182efff7efe74024b8ec15d50b6e5f2c/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:17.502750 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.503125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.503466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.551227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] MFA Rules not processed for user `182efff7efe74024b8ec15d50b6e5f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:17.552445 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:17.585181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:17.634537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1242/2484] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:17.646776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:17.648653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:17.733826 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31c3c-cbbc-4b66-af4d-ce7b051f27b2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:17.738091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cddbb-8a07-4f06-bb34-5bb956c53f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:17.738810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cddbb-8a07-4f06-bb34-5bb956c53f3d service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:17.739113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cddbb-8a07-4f06-bb34-5bb956c53f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:17.739429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cddbb-8a07-4f06-bb34-5bb956c53f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:17.750564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cddbb-8a07-4f06-bb34-5bb956c53f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:17.752153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cddbb-8a07-4f06-bb34-5bb956c53f3d service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:17.765566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1243/2485] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:20.375483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:20.441938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b94445e-56cb-478a-9a11-74bda82d0d43 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:20.447424 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:20.448341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:20.448633 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:20.448901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] PATH_INFO: `/v3/users/182efff7efe74024b8ec15d50b6e5f2c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:20.450542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=182efff7efe74024b8ec15d50b6e5f2c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:20.452467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:20.507543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182efff7efe74024b8ec15d50b6e5f2c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:20.508192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182efff7efe74024b8ec15d50b6e5f2c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:20.520990 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182efff7efe74024b8ec15d50b6e5f2c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:20.528173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182efff7efe74024b8ec15d50b6e5f2c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:20.535453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182efff7efe74024b8ec15d50b6e5f2c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:20.541668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1243/2486] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:20 2022] DELETE /identity/v3/users/182efff7efe74024b8ec15d50b6e5f2c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:20.546229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:20.546734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:20.546855 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:20.566152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:20.567284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:20.630250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1244/2487] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:20.640535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:20.641969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:20.709915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:20.730745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeba9b1-d7ae-49dc-a8a7-4b96c9d0c7bd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:20.736109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e79e99-506d-49c5-82d8-3eb5f005fa07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:20.736981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79e99-506d-49c5-82d8-3eb5f005fa07 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:20.737673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79e99-506d-49c5-82d8-3eb5f005fa07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:20.737673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e79e99-506d-49c5-82d8-3eb5f005fa07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:20.750126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e79e99-506d-49c5-82d8-3eb5f005fa07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:20.751988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e79e99-506d-49c5-82d8-3eb5f005fa07 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:20.764723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1244/2488] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:21.166682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.168419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:21.184941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94430f9-b351-4500-a32e-81fd014d2236 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.192540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.193637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.194033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.194419 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] PATH_INFO: `/v3/projects/011891ff0c69442eafd640a97a958552` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.196098 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=011891ff0c69442eafd640a97a958552)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.199198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.245863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:21.247397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '011891ff0c69442eafd640a97a958552', 'request_id': 'req-ab7da129-dbd1-46aa-bd42-3baf39617687'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:21.255660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '011891ff0c69442eafd640a97a958552', 'request_id': 'req-ab7da129-dbd1-46aa-bd42-3baf39617687'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:21.261532 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '011891ff0c69442eafd640a97a958552', 'request_id': 'req-ab7da129-dbd1-46aa-bd42-3baf39617687'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:21.273580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] The token cache is being invalidate because project 011891ff0c69442eafd640a97a958552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:21.273861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:21.274464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ab7da129-dbd1-46aa-bd42-3baf39617687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:21.276651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1245/2489] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:21 2022] DELETE /identity/v3/projects/011891ff0c69442eafd640a97a958552 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:21.281540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.281821 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.282091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.301689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:21.302888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:21.367243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1245/2490] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:21.375856 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.376136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.376410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.396160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:21.397387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:21.406980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1246/2491] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:21.411005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.421448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd59446-ff3a-4237-af94-4e03e8f54fbe None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.426033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.426639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.426918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.427190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.427885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:21.428184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.429815 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.441585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1246/2492] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:20:21.445879 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.455215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753f8cf-7c32-4226-a6c9-53864e973598 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.459840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.460441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.460718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.461007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.461670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.464106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.492126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1247/2493] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:21 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:20:21.496509 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.506625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23431d7-41ae-4870-8011-f538d5da4157 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.566293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.566981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.567192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.567460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.568123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.570066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.597261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1247/2494] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:21 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:20:21.598613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.610159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe01ea6-9e39-4a91-a99c-4baf617a103a admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.615485 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.616131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.616433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.616734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.617541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.619154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.631082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1248/2495] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:21.636853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.647882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ffbaaa-5f86-411b-b1d1-b8ebff80c379 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.654568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75140eb3-9b26-41c9-8f0b-f8037bc57f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.655551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75140eb3-9b26-41c9-8f0b-f8037bc57f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.655551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75140eb3-9b26-41c9-8f0b-f8037bc57f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.655963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75140eb3-9b26-41c9-8f0b-f8037bc57f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.656675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75140eb3-9b26-41c9-8f0b-f8037bc57f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.658338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75140eb3-9b26-41c9-8f0b-f8037bc57f0a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.670757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1248/2496] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:20:21.675068 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.686793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a4d9b-455a-4a30-8ca3-64afbb7dc228 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.691943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.692569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.692853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.693164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] PATH_INFO: `/v3/projects/6abea21d51da414d804056066647a6fa/users/8af9214eabc743c28bb3b8555ac70596/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.711425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6abea21d51da414d804056066647a6fa, user_id=8af9214eabc743c28bb3b8555ac70596, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.714209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.728480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1249/2497] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:21 2022] PUT /identity/v3/projects/6abea21d51da414d804056066647a6fa/users/8af9214eabc743c28bb3b8555ac70596/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:21.740457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92b0d-8412-469a-bdc6-6aa2d34bbfe2 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.740457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92b0d-8412-469a-bdc6-6aa2d34bbfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.740457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a92b0d-8412-469a-bdc6-6aa2d34bbfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.770166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f9a92b0d-8412-469a-bdc6-6aa2d34bbfe2 None None] MFA Rules not processed for user `8af9214eabc743c28bb3b8555ac70596`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:21.771527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f9a92b0d-8412-469a-bdc6-6aa2d34bbfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:21.833563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:21.873106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1249/2498] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:21.885216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:21.887018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:21.959162 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348642a6-0748-4d64-a159-68310c02d7ba admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:21.964034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:21.964678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:21.964983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:21.965283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:21.975038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:21.976654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:21.988247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1250/2499] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:27.691140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:27.691605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:27.691728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:27.714368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:27.715596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:27.771531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1250/2500] 10.222.0.48 () {58 vars in 967 bytes} [Tue Jul 26 17:20:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:27.789697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:27.791399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:27.872472 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8489-ae80-4144-b555-ff13f6c22877 service glance] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:27.878078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59705531-3353-420a-be96-a227461bfe76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:27.878712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59705531-3353-420a-be96-a227461bfe76 service swift] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:27.879115 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59705531-3353-420a-be96-a227461bfe76 service swift] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:27.879317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59705531-3353-420a-be96-a227461bfe76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:27.889279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59705531-3353-420a-be96-a227461bfe76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:27.890906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59705531-3353-420a-be96-a227461bfe76 service swift] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:27.901051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1251/2501] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:20:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:20:28.390035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:28.391586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:28.459301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3a8d-c7b3-4a74-9f45-0b26df0c7370 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:28.465557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:28.466955 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:28.467202 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:28.467584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:28.470153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:28.545791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:28.547270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:28.557124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1251/2502] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:32.456452 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59705531-3353-420a-be96-a227461bfe76 service swift] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:32.458944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-59705531-3353-420a-be96-a227461bfe76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:32.476080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:32.482492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59705531-3353-420a-be96-a227461bfe76 service swift] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:32.489828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:32.490542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:32.491110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:32.491849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] PATH_INFO: `/v3/users/459db7ff580948dc853b9f843700fe3c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:32.493431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=459db7ff580948dc853b9f843700fe3c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:32.497756 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:32.580549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459db7ff580948dc853b9f843700fe3c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:32.594103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459db7ff580948dc853b9f843700fe3c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:32.603784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459db7ff580948dc853b9f843700fe3c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:32.604776 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459db7ff580948dc853b9f843700fe3c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:32.605449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7e6ed44b-3479-4c5f-85a2-3d1596b925ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459db7ff580948dc853b9f843700fe3c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:32.631534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1252/2503] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:32 2022] DELETE /identity/v3/users/459db7ff580948dc853b9f843700fe3c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:32.930481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:32.940857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5230f3-6abc-4abb-80f1-8138043ac8a6 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:32.993986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:33.019167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.019822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.020104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.020373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] PATH_INFO: `/v3/projects/6f7fbc938d614078b719aa483aa80cd6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.021577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7fbc938d614078b719aa483aa80cd6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.023487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.078789 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7fbc938d614078b719aa483aa80cd6', 'request_id': 'req-7748c27a-4543-46e7-acb7-9c0fde151e99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:33.091076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7fbc938d614078b719aa483aa80cd6', 'request_id': 'req-7748c27a-4543-46e7-acb7-9c0fde151e99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:33.098508 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7fbc938d614078b719aa483aa80cd6', 'request_id': 'req-7748c27a-4543-46e7-acb7-9c0fde151e99'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:33.109906 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] The token cache is being invalidate because project 6f7fbc938d614078b719aa483aa80cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:33.110192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:33.110853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7748c27a-4543-46e7-acb7-9c0fde151e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:33.112684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1252/2504] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:32 2022] DELETE /identity/v3/projects/6f7fbc938d614078b719aa483aa80cd6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:33.119490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.119769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.120069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.141777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:33.142982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:33.204412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1253/2505] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:33.212426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.212707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.213073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.233344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:33.234551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:33.244105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1253/2506] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:33.248357 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.259495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca8315-afa5-48dd-9b8e-8e91f0a2bb77 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.265661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.266209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.266478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.266795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.267509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:33.267819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.269474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.282133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1254/2507] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:20:33.286422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.303141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ea235-9758-4196-9c9b-2be26b1dd359 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.308752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.309466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.309744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.310019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.310678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.312584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.342293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1254/2508] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:33.346499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.357943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301955d-936c-4e47-bef3-f286f940c873 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.417391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.418019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.418303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.418576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.419244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.421235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.432033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:33.445002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1255/2509] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:20:33.449742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.460337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57368bfa-ab35-4f74-be6b-53be8fdda874 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.465564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.466175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.466455 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.466731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.467478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.469391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.485353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1255/2510] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:33.489957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.501422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548219b-434f-4448-9439-bfeb7038da03 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.506180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34846959-0fdf-4d5e-80c5-34b5f246b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.506774 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846959-0fdf-4d5e-80c5-34b5f246b6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.507055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846959-0fdf-4d5e-80c5-34b5f246b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.507329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34846959-0fdf-4d5e-80c5-34b5f246b6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.508019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846959-0fdf-4d5e-80c5-34b5f246b6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.509677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34846959-0fdf-4d5e-80c5-34b5f246b6d5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.520182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1256/2511] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:33.524304 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.534983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050cdd4-b4e1-4129-a21b-21b070492e53 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.540184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.540952 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.541275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.541524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] PATH_INFO: `/v3/projects/e9ddf51573f046b4bfa19f8a478e0cec/users/00880335a3094d018c2828f358971ec0/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.557276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ddf51573f046b4bfa19f8a478e0cec, user_id=00880335a3094d018c2828f358971ec0, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.559985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.572853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1256/2512] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:33 2022] PUT /identity/v3/projects/e9ddf51573f046b4bfa19f8a478e0cec/users/00880335a3094d018c2828f358971ec0/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:33.584275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.584572 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.584946 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.612890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] MFA Rules not processed for user `00880335a3094d018c2828f358971ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:33.614191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:33.682372 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1257/2513] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:33.694143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.763079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a29ac-577f-478e-aed4-c292394a6761 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.767971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.768596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.768878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.769214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.769977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.771891 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.778312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:33.812845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1257/2514] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:33.813987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.825447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7392f3a-bf82-4804-9cd4-6f2b2e24c6c6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.884026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.884790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.885135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.885438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.886119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.888066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.908410 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1258/2515] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:33 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:20:33.912533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.923067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647e0d-427b-436f-bef5-4000ff685904 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.927735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.928343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.928625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.928901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.929631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.931770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.941741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1258/2516] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:33.945895 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:33.956789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51954779-13da-4c52-a9e9-7ae7254043a1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:33.961722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:33.962386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:33.962667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:33.962945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] PATH_INFO: `/v3/projects/ac36d390a5194ccf89caf82b3461882d/users/09a058db02944d5eb5c8e788191e6838/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:33.979476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac36d390a5194ccf89caf82b3461882d, user_id=09a058db02944d5eb5c8e788191e6838, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:33.982310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:33.997666 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1259/2517] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:33 2022] PUT /identity/v3/projects/ac36d390a5194ccf89caf82b3461882d/users/09a058db02944d5eb5c8e788191e6838/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:34.002027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:34.012838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1cce0-07af-400a-b063-a3dea396100f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:34.072840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4509ba-e080-4781-94ac-a59cf07c6778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:34.073491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4509ba-e080-4781-94ac-a59cf07c6778 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:34.073776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4509ba-e080-4781-94ac-a59cf07c6778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:34.074053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4509ba-e080-4781-94ac-a59cf07c6778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:34.074810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4509ba-e080-4781-94ac-a59cf07c6778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:34.076474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4509ba-e080-4781-94ac-a59cf07c6778 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:34.090661 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1259/2518] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:20:34.095502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:34.107065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0fe19-ebef-4ae5-9d9d-1a63170367c6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:34.115534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:34.116220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:34.116500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:34.117471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] PATH_INFO: `/v3/projects/ac36d390a5194ccf89caf82b3461882d/users/09a058db02944d5eb5c8e788191e6838/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:34.122304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac36d390a5194ccf89caf82b3461882d, user_id=09a058db02944d5eb5c8e788191e6838, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:34.125332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:34.136762 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1260/2519] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:34 2022] PUT /identity/v3/projects/ac36d390a5194ccf89caf82b3461882d/users/09a058db02944d5eb5c8e788191e6838/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:34.147049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:34.147485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:34.147705 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:34.179659 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] MFA Rules not processed for user `09a058db02944d5eb5c8e788191e6838`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:34.181872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:34.252529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1260/2520] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:34.263559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:34.265573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:34.294583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:34.327214 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea02088-e88f-4710-9a18-34961dc48bdf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:34.330933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ee6a8-8d62-498c-a289-23155f2c02df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:34.331706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ee6a8-8d62-498c-a289-23155f2c02df service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:34.331986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ee6a8-8d62-498c-a289-23155f2c02df service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:34.332253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ee6a8-8d62-498c-a289-23155f2c02df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:34.342003 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ee6a8-8d62-498c-a289-23155f2c02df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:34.343443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ee6a8-8d62-498c-a289-23155f2c02df service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:34.353884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1261/2521] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:34.460297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:34.470094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da1ea9-722c-4eab-88bd-efd76e7ad43c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:34.473847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ab278-40ce-41b9-afc6-ec652b021edb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:34.474484 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ab278-40ce-41b9-afc6-ec652b021edb service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:34.474761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ab278-40ce-41b9-afc6-ec652b021edb service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:34.475029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ab278-40ce-41b9-afc6-ec652b021edb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:34.532022 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:34.534759 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ab278-40ce-41b9-afc6-ec652b021edb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:34.536397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ab278-40ce-41b9-afc6-ec652b021edb service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:34.547170 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1261/2522] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:35.578569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea007e6-b8c7-4e54-bd48-c13e24ef4401 None None] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:35.579129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea007e6-b8c7-4e54-bd48-c13e24ef4401 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:35.579475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea007e6-b8c7-4e54-bd48-c13e24ef4401 None None] PATH_INFO: `/` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:35.582948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1262/2523] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:20:35.590704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d08ec-bf8a-477d-bfbf-9b536bc856f4 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:35.591174 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d08ec-bf8a-477d-bfbf-9b536bc856f4 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:35.595517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d08ec-bf8a-477d-bfbf-9b536bc856f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:35.643071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c62d08ec-bf8a-477d-bfbf-9b536bc856f4 None None] MFA Rules not processed for user `ed1fb62d8f3f4a73ac9d11a37c62f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:35.644710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c62d08ec-bf8a-477d-bfbf-9b536bc856f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:35.749783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1262/2524] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:20:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:35.767260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea007e6-b8c7-4e54-bd48-c13e24ef4401 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:35.769304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fea007e6-b8c7-4e54-bd48-c13e24ef4401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:35.786731 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea007e6-b8c7-4e54-bd48-c13e24ef4401 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:35.791579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9343796-8a8b-4f9c-bac9-418cc3be33fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:35.793184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9343796-8a8b-4f9c-bac9-418cc3be33fa service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:35.793722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9343796-8a8b-4f9c-bac9-418cc3be33fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:35.794191 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9343796-8a8b-4f9c-bac9-418cc3be33fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:35.802884 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9343796-8a8b-4f9c-bac9-418cc3be33fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:35.804660 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9343796-8a8b-4f9c-bac9-418cc3be33fa service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:35.829715 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1263/2525] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:35.922402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045828aa-4df7-4d2f-801d-7e7361945f73 None None] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:35.922741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045828aa-4df7-4d2f-801d-7e7361945f73 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:35.923135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045828aa-4df7-4d2f-801d-7e7361945f73 None None] PATH_INFO: `/` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:35.925428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1263/2526] 10.222.0.48 () {54 vars in 921 bytes} [Tue Jul 26 17:20:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 17:20:35.932543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:35.933287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:35.933594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:35.954615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] MFA Rules not processed for user `a903f3771a62447d9c06fbeba414e9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:35.955836 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:36.042718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1264/2527] 10.222.0.48 () {58 vars in 1018 bytes} [Tue Jul 26 17:20:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:36.053814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045828aa-4df7-4d2f-801d-7e7361945f73 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:36.077407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045828aa-4df7-4d2f-801d-7e7361945f73 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:36.081344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ccba6-2bbd-4b6b-92f3-75d6355693fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:36.081964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ccba6-2bbd-4b6b-92f3-75d6355693fa service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:36.082247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ccba6-2bbd-4b6b-92f3-75d6355693fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:36.082520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ccba6-2bbd-4b6b-92f3-75d6355693fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:36.118159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ccba6-2bbd-4b6b-92f3-75d6355693fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:36.119768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ccba6-2bbd-4b6b-92f3-75d6355693fa service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:36.147957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1264/2528] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:38.306012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:38.380517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1220360-6afb-4f5b-b0d8-0f841cf803b3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:38.386462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:38.387331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:38.387800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:38.388243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] PATH_INFO: `/v3/users/00880335a3094d018c2828f358971ec0` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:38.389905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00880335a3094d018c2828f358971ec0)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:38.393773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:38.448652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00880335a3094d018c2828f358971ec0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:38.449914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:38.458764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00880335a3094d018c2828f358971ec0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:38.466801 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00880335a3094d018c2828f358971ec0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:38.467488 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00880335a3094d018c2828f358971ec0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:38.468176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00880335a3094d018c2828f358971ec0'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:38.488260 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1265/2529] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:38 2022] DELETE /identity/v3/users/00880335a3094d018c2828f358971ec0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:38.492872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:38.493447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:38.493447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:38.513193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:38.514680 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:38.602926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1265/2530] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:38.613498 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:38.614981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:38.675594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812410e1-0fa0-4b79-b2f1-761539cac598 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:38.679235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:38.679862 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:38.680139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:38.680408 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:38.693853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:38.695375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:38.706484 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1266/2531] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:39.085853 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:39.087347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:39.100074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23191098-8c07-42da-87f4-2e58deb63ba3 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:39.105513 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:39.106158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:39.106477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:39.106777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] PATH_INFO: `/v3/projects/e9ddf51573f046b4bfa19f8a478e0cec` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:39.107960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ddf51573f046b4bfa19f8a478e0cec)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:39.110131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:39.178893 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ddf51573f046b4bfa19f8a478e0cec', 'request_id': 'req-6170aa08-8d91-4e34-aecc-6464a8075d6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.186204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ddf51573f046b4bfa19f8a478e0cec', 'request_id': 'req-6170aa08-8d91-4e34-aecc-6464a8075d6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.197448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ddf51573f046b4bfa19f8a478e0cec', 'request_id': 'req-6170aa08-8d91-4e34-aecc-6464a8075d6d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.215290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] The token cache is being invalidate because project e9ddf51573f046b4bfa19f8a478e0cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:39.215583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.216229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.217968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1266/2532] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:39 2022] DELETE /identity/v3/projects/e9ddf51573f046b4bfa19f8a478e0cec => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:39.221792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:39.234827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f864da2-4687-42f1-a7fb-715ebbf02a4c service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:39.317524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:39.318350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:39.318699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:39.319036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] PATH_INFO: `/v3/users/09a058db02944d5eb5c8e788191e6838` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:39.320369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a058db02944d5eb5c8e788191e6838)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:39.323494 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:39.390711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a058db02944d5eb5c8e788191e6838'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.398462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a058db02944d5eb5c8e788191e6838'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.409857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a058db02944d5eb5c8e788191e6838'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.410590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a058db02944d5eb5c8e788191e6838'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.411202 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-65734447-3023-4da3-9de7-119e2b365a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a058db02944d5eb5c8e788191e6838'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.417284 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:39.432839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1267/2533] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:39 2022] DELETE /identity/v3/users/09a058db02944d5eb5c8e788191e6838 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:39.702613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:39.704636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:39.719102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170aa08-8d91-4e34-aecc-6464a8075d6d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:39.731540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:39.786924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:39.787575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:39.787886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:39.788166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] PATH_INFO: `/v3/projects/ac36d390a5194ccf89caf82b3461882d` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:39.789386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac36d390a5194ccf89caf82b3461882d)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:39.791601 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:39.840076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac36d390a5194ccf89caf82b3461882d', 'request_id': 'req-e1dfe80c-a727-424f-aac5-629db6c16d7c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.848143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac36d390a5194ccf89caf82b3461882d', 'request_id': 'req-e1dfe80c-a727-424f-aac5-629db6c16d7c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.856117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac36d390a5194ccf89caf82b3461882d', 'request_id': 'req-e1dfe80c-a727-424f-aac5-629db6c16d7c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.866761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] The token cache is being invalidate because project ac36d390a5194ccf89caf82b3461882d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:39.867046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.867697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-e1dfe80c-a727-424f-aac5-629db6c16d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:39.870004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1267/2534] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:39 2022] DELETE /identity/v3/projects/ac36d390a5194ccf89caf82b3461882d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:39.875341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:39.875797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:39.876088 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:39.897754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:39.898962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:39.962581 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1268/2535] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:39.971000 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:39.971287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:39.971563 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:39.999014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:40.000282 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:40.010971 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1268/2536] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:40.015594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.026269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4332ac-2c33-4b2d-aa70-22486674bea0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.031615 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.032236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.032515 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.032787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.033585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:40.033896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.035514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.048522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1269/2537] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:20:40.053094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.065840 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ce92-50c8-4b05-9311-a957b3d85ed9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.074229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.075173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.075457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.075728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.076403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.078347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.107383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1269/2538] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:40 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:40.111680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.123504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745158e6-d52c-4a04-aeef-2cd325644602 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.180668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.182497 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.182787 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.183063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.183737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.185689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.208263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1270/2539] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:40 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:40.212433 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.224907 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5a6e-ac16-4d92-95bd-140a0b5cad74 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.230335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.231105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.231389 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.231668 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.232381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.234168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.245359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1270/2540] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:40.249740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.260207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107b54f-d6fa-4d33-91ee-d3c947b8d988 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.267900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac7520-eb5c-4727-a940-bafd3072410b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.268526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7520-eb5c-4727-a940-bafd3072410b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.268804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7520-eb5c-4727-a940-bafd3072410b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.269109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac7520-eb5c-4727-a940-bafd3072410b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.269798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac7520-eb5c-4727-a940-bafd3072410b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.271413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac7520-eb5c-4727-a940-bafd3072410b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.283802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1271/2541] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:40.288023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.299486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f2d9a-2d4c-4699-8c07-3bfeb8fe5176 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.311193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.311976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.312706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.313089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] PATH_INFO: `/v3/projects/02ea788020be400ea141e8abeb59f6ac/users/35eee5f7bf84433793792f9d70b28e18/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.336632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ea788020be400ea141e8abeb59f6ac, user_id=35eee5f7bf84433793792f9d70b28e18, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.339901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.366195 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1271/2542] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:40 2022] PUT /identity/v3/projects/02ea788020be400ea141e8abeb59f6ac/users/35eee5f7bf84433793792f9d70b28e18/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:40.378204 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.378753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.378850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.417473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] MFA Rules not processed for user `35eee5f7bf84433793792f9d70b28e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:40.418925 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:40.497182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:40.530233 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1272/2543] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:40.535100 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.598295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:40.622011 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6775b-e546-4a70-90b8-5579ecd209ed admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.628244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.629089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.629467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.629965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.630848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.633766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.680623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1272/2544] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:40 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:40.685268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.699811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9fc29b-dbb2-4ed0-8558-df9281f18ebe None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.785953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.786742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.787026 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.787316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.788033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.790222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.818512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1273/2545] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:40 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:40.823435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.834699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa47e20-02dd-408e-966d-20298a1a3a5f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.842761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.843296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.843577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.843851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.844744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.846419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.860019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1273/2546] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:40.864671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.881659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ee90-dbcc-4b49-86f7-7a6d3e2d2269 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:40.888900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:40.889628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:40.889909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:40.890184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] PATH_INFO: `/v3/projects/153417d829c24d58bdbe9f04e67670fa/users/3ed4f5ed8f544434a32b1bfcfaa898a6/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:40.912428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=153417d829c24d58bdbe9f04e67670fa, user_id=3ed4f5ed8f544434a32b1bfcfaa898a6, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:40.917506 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:40.934119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1274/2547] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:40 2022] PUT /identity/v3/projects/153417d829c24d58bdbe9f04e67670fa/users/3ed4f5ed8f544434a32b1bfcfaa898a6/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:20:40.939851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:40.951810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddabaf6-edd3-49b7-8e41-e048cc92d133 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:41.013802 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347f00-de43-4515-8063-3cf3939df576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:41.014422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347f00-de43-4515-8063-3cf3939df576 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:41.014699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347f00-de43-4515-8063-3cf3939df576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:41.014972 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347f00-de43-4515-8063-3cf3939df576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:41.015665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67347f00-de43-4515-8063-3cf3939df576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:41.017265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67347f00-de43-4515-8063-3cf3939df576 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:41.031417 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1274/2548] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:41.032656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:41.044082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609a6cf-327d-4035-b0db-9e350c3ad20e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:41.048902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:41.049632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:41.049916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:41.050198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] PATH_INFO: `/v3/projects/153417d829c24d58bdbe9f04e67670fa/users/3ed4f5ed8f544434a32b1bfcfaa898a6/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:41.052343 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=153417d829c24d58bdbe9f04e67670fa, user_id=3ed4f5ed8f544434a32b1bfcfaa898a6, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:41.055241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:41.066669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1275/2549] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:41 2022] PUT /identity/v3/projects/153417d829c24d58bdbe9f04e67670fa/users/3ed4f5ed8f544434a32b1bfcfaa898a6/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:41.078288 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:41.078581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:41.078858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:41.117978 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] MFA Rules not processed for user `3ed4f5ed8f544434a32b1bfcfaa898a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:41.119994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:41.198940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1275/2550] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:41.210308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:41.213125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:41.273839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b776f-3a57-43c9-a6b2-6472d4f16cfb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:41.278458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b723d0-4318-4be4-8582-5baa4c8d5586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:41.279092 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b723d0-4318-4be4-8582-5baa4c8d5586 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:41.279373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b723d0-4318-4be4-8582-5baa4c8d5586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:41.279647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b723d0-4318-4be4-8582-5baa4c8d5586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:41.334966 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:41.347008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b723d0-4318-4be4-8582-5baa4c8d5586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:41.352177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b723d0-4318-4be4-8582-5baa4c8d5586 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:41.363644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1276/2551] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:43.297996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:43.300493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:43.358069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:43.374754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f14de-1b18-4948-9916-d0a4f7aa007b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:43.378412 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:43.379034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:43.379311 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:43.379580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:43.381379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:43.457443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:43.459074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:43.472571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1276/2552] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:44.172872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:44.172872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:44.172872 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:44.192585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] MFA Rules not processed for user `dcf968aeb53c4215927642e9931b4a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:44.193849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:44.250765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1277/2553] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:47.025956 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:47.027501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:47.047910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4cb4e7-a7bf-47e4-91ad-ba1e7973a620 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:47.052911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:47.053628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.053911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.054191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.064034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:47.066125 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:47.066953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:47.068564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:47.079774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1277/2554] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:47.145725 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:47.146260 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:47.161554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b327789-195c-4aa7-b9a3-882f4e62ecb4 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:47.167166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:47.168252 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.168539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.169203 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] PATH_INFO: `/v3/users/c6291932ceb84b03924c0f056f29c2a9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.171431 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6291932ceb84b03924c0f056f29c2a9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:47.173809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:47.177779 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d99034-e624-4e6c-847d-d03183eac043 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:47.184062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:47.184882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.185359 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.185802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.187985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:47.202970 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:47.203864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:47.216455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1278/2555] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:20:47.249732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6291932ceb84b03924c0f056f29c2a9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.251046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6291932ceb84b03924c0f056f29c2a9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.272480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6291932ceb84b03924c0f056f29c2a9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.286106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6291932ceb84b03924c0f056f29c2a9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.300443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-15b004fd-8294-4da7-8ebb-14a0b17b652a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6291932ceb84b03924c0f056f29c2a9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.312074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1278/2556] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:47 2022] DELETE /identity/v3/users/c6291932ceb84b03924c0f056f29c2a9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:20:47.563050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:47.576750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f95cf-b1f4-443c-a735-20e365cc0ed3 service placement] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:47.661512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:47.662245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.662527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.662830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] PATH_INFO: `/v3/projects/446eaae0a4f34eba86d4b0de0c2d1013` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.664047 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=446eaae0a4f34eba86d4b0de0c2d1013)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:47.667240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:47.703269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:47.723645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '446eaae0a4f34eba86d4b0de0c2d1013', 'request_id': 'req-49e24805-ea35-470f-998b-be88ea8c7db5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.732702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '446eaae0a4f34eba86d4b0de0c2d1013', 'request_id': 'req-49e24805-ea35-470f-998b-be88ea8c7db5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.742168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '446eaae0a4f34eba86d4b0de0c2d1013', 'request_id': 'req-49e24805-ea35-470f-998b-be88ea8c7db5'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.757561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] The token cache is being invalidate because project 446eaae0a4f34eba86d4b0de0c2d1013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:47.758049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.758698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49e24805-ea35-470f-998b-be88ea8c7db5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:47.760413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1279/2557] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:47 2022] DELETE /identity/v3/projects/446eaae0a4f34eba86d4b0de0c2d1013 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:47.766920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.767207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.767535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.788280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:47.789524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:47.834029 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:47.855552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1279/2558] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:47.863764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.864053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.864446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.886545 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:47.887986 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:47.897066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1280/2559] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:47.901511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:47.911276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc9943-d813-469b-a563-287cba197e6d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:47.917791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:47.918493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.918838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.919178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.919958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:47.920323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:47.921945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:47.935654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1280/2560] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:20:47.939243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:47.949108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c8f3-533f-4112-a122-3ae071b18d61 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:47.954350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:47.954861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:47.955311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:47.955589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:47.956273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:47.958234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:47.991030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1281/2561] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:47 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:47.995965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.006809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd453a4b-d697-40bd-ab44-9095b3cae5db admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.079753 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.081190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.081988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.081988 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.082609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.084623 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.122746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1281/2562] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:47 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:48.124703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.160032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa4ba0-02e3-4bf1-9a9f-51a971906a66 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.168318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.169039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.169390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.170445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.171268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.174505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.199211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1282/2563] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:48.200574 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.220464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410b0e7-ce68-45b2-bf53-941b53a9db8e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.229927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568ac6-a472-4e20-b36c-9351c2ceff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.230752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568ac6-a472-4e20-b36c-9351c2ceff0c admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.231186 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568ac6-a472-4e20-b36c-9351c2ceff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.231634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568ac6-a472-4e20-b36c-9351c2ceff0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.232488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14568ac6-a472-4e20-b36c-9351c2ceff0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.234591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14568ac6-a472-4e20-b36c-9351c2ceff0c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.250785 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1282/2564] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:20:48.255320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.269686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a21b27-b64e-440d-9b02-b428ff728111 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.275220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.275903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.276234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.276827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] PATH_INFO: `/v3/projects/62d38e80bd1449d5bb0a23feaeae54f5/users/1f959744cfc74ea2ab1d0af35109d804/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.295957 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d38e80bd1449d5bb0a23feaeae54f5, user_id=1f959744cfc74ea2ab1d0af35109d804, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.298824 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.315425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1283/2565] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:48 2022] PUT /identity/v3/projects/62d38e80bd1449d5bb0a23feaeae54f5/users/1f959744cfc74ea2ab1d0af35109d804/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:48.325693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.325983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.326261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.367819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] MFA Rules not processed for user `1f959744cfc74ea2ab1d0af35109d804`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:48.369342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:48.458098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1283/2566] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:48.463307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.558865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290cd0-4979-4fb6-a087-e7e27c962542 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.580062 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.581683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.582183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.582905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.584187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.586699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.623141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1284/2567] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:48 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:48.627177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.645197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db8915-194e-4005-bbe1-8b28335b87ca None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.703345 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:48.724027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.724783 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.725221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.725589 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.726367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.729383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.753078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1284/2568] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:48 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:48.757206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.777943 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef32202-46c8-4960-a39c-d90294b80d6d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.786008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.786638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.786949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.787275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.787914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.789642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.801792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1285/2569] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:20:48.806698 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.818051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099456d-6003-40b3-a416-30a3de4e0939 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.826565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.827517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.827810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.828094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] PATH_INFO: `/v3/projects/91372fdcaee142ae97dc3c481cc08c27/users/590145cb2b044edbb3be560023bff541/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.845539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91372fdcaee142ae97dc3c481cc08c27, user_id=590145cb2b044edbb3be560023bff541, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.848340 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.863095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1285/2570] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:48 2022] PUT /identity/v3/projects/91372fdcaee142ae97dc3c481cc08c27/users/590145cb2b044edbb3be560023bff541/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:20:48.867337 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.881216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c359a23-f117-4f7a-a7f4-6042ba3c2625 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.898307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:48.939850 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b178495-af04-48bc-b2f7-5e99b7aefe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.940471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178495-af04-48bc-b2f7-5e99b7aefe5f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.940754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178495-af04-48bc-b2f7-5e99b7aefe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.941069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b178495-af04-48bc-b2f7-5e99b7aefe5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.941758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b178495-af04-48bc-b2f7-5e99b7aefe5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.943355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b178495-af04-48bc-b2f7-5e99b7aefe5f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.954024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1286/2571] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:48.958368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:48.969333 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141c14c-c630-4fd0-a942-75266bbf5560 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:48.974030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:48.974628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:48.974933 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:48.975207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] PATH_INFO: `/v3/projects/91372fdcaee142ae97dc3c481cc08c27/users/590145cb2b044edbb3be560023bff541/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:48.977216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91372fdcaee142ae97dc3c481cc08c27, user_id=590145cb2b044edbb3be560023bff541, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:48.980467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:48.990799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1286/2572] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:48 2022] PUT /identity/v3/projects/91372fdcaee142ae97dc3c481cc08c27/users/590145cb2b044edbb3be560023bff541/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:49.002734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:49.003411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:49.003411 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:49.033138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] MFA Rules not processed for user `590145cb2b044edbb3be560023bff541`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:49.034742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:49.102788 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1287/2573] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:49.115675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:49.117507 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:49.178274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad0150-1b96-450c-8964-b21eae123592 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:49.182572 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af50fcd-5b43-4619-b260-f9c4fe9dec70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:49.183187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af50fcd-5b43-4619-b260-f9c4fe9dec70 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:49.183469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af50fcd-5b43-4619-b260-f9c4fe9dec70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:49.183742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af50fcd-5b43-4619-b260-f9c4fe9dec70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:49.193866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af50fcd-5b43-4619-b260-f9c4fe9dec70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:49.195301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af50fcd-5b43-4619-b260-f9c4fe9dec70 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:49.209143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1287/2574] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:49.241930 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:49.319983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2b584-163f-47a5-9c1b-4f9af8ed2a8b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:49.325928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:49.326677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:49.327969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:49.328340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] PATH_INFO: `/v3/users/1f959744cfc74ea2ab1d0af35109d804` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:49.329737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f959744cfc74ea2ab1d0af35109d804)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:49.331720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:49.408008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f959744cfc74ea2ab1d0af35109d804'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:49.409060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:49.418028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f959744cfc74ea2ab1d0af35109d804'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:49.427328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f959744cfc74ea2ab1d0af35109d804'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:49.428139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f959744cfc74ea2ab1d0af35109d804'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:49.428720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f959744cfc74ea2ab1d0af35109d804'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:49.448350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1288/2575] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:49 2022] DELETE /identity/v3/users/1f959744cfc74ea2ab1d0af35109d804 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:49.453734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:49.454025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:49.454334 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:49.486280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:49.487463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:49.547013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1288/2576] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:49.556589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:49.558032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:49.615567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63dbe7-ba61-451f-b6b4-1af17498859f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:49.619840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:49.620454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:49.620738 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:49.621036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:49.630622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:49.632058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:49.642286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1289/2577] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:49.928377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:49.930300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:49.953079 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0ef61-a833-4dc1-a007-01400a043b88 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:49.959597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:49.960614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:49.960894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:49.961217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] PATH_INFO: `/v3/projects/62d38e80bd1449d5bb0a23feaeae54f5` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:49.963338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d38e80bd1449d5bb0a23feaeae54f5)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:49.965694 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:50.020432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:50.037023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d38e80bd1449d5bb0a23feaeae54f5', 'request_id': 'req-51eb4b6b-4637-4e28-be17-e9823c90839d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.045299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d38e80bd1449d5bb0a23feaeae54f5', 'request_id': 'req-51eb4b6b-4637-4e28-be17-e9823c90839d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.052104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d38e80bd1449d5bb0a23feaeae54f5', 'request_id': 'req-51eb4b6b-4637-4e28-be17-e9823c90839d'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.063403 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] The token cache is being invalidate because project 62d38e80bd1449d5bb0a23feaeae54f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:50.063686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.064336 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.066030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1289/2578] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:49 2022] DELETE /identity/v3/projects/62d38e80bd1449d5bb0a23feaeae54f5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:50.070134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:50.080028 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118471b-0d71-4983-b56c-e93be65c85f8 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:50.135483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:50.136124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:50.136402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:50.136675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] PATH_INFO: `/v3/users/590145cb2b044edbb3be560023bff541` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:50.137867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=590145cb2b044edbb3be560023bff541)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:50.139782 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:50.186305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590145cb2b044edbb3be560023bff541'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.193066 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590145cb2b044edbb3be560023bff541'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.201780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590145cb2b044edbb3be560023bff541'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.202423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590145cb2b044edbb3be560023bff541'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.203019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-7cdca13c-9278-417f-aa59-26fc9d80a96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590145cb2b044edbb3be560023bff541'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.208504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:50.223369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1290/2579] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:50 2022] DELETE /identity/v3/users/590145cb2b044edbb3be560023bff541 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:50.539398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:50.542309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:50.569981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4b6b-4637-4e28-be17-e9823c90839d admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:50.637073 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:50.637805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:50.638154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:50.638478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] PATH_INFO: `/v3/projects/91372fdcaee142ae97dc3c481cc08c27` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:50.639701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91372fdcaee142ae97dc3c481cc08c27)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:50.641805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:50.691594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91372fdcaee142ae97dc3c481cc08c27', 'request_id': 'req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.699475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91372fdcaee142ae97dc3c481cc08c27', 'request_id': 'req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.705770 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91372fdcaee142ae97dc3c481cc08c27', 'request_id': 'req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.716895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] The token cache is being invalidate because project 91372fdcaee142ae97dc3c481cc08c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:50.717217 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.717864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-95e1440a-48cf-445f-ad56-a8ce9c9c5ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:50.719381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1290/2580] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:50 2022] DELETE /identity/v3/projects/91372fdcaee142ae97dc3c481cc08c27 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:20:50.725123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:50.725598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:50.725711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:50.747084 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:50.748208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:50.805325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1291/2581] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:50.813596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:50.813886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:50.814166 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:50.835339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:50.836578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:50.846427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1291/2582] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:50.850854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:50.861535 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3621bb0-fbdc-45cd-a367-5ed2d26ea7b8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:50.866138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:50.866752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:50.867086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:50.867305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:50.868091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:50.869048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:50.870621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:50.882651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1292/2583] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:20:50.887595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:50.897443 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ce7d4-5c1a-4bdc-8096-845e9737f305 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:50.902419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:50.903036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:50.903321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:50.903606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:50.904415 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:50.906204 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:50.934771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1292/2584] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:50 2022] POST /identity/v3/projects => generated 407 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:50.939016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:50.949806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f42d1-0a38-4d35-9677-90e0ce563a40 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.012693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.013351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.013632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.013904 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.014575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.016501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.041014 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1293/2585] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:50 2022] POST /identity/v3/users => generated 304 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:51.044786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.055606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d2fc6-0742-4054-96a4-91538ec0dd15 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.061765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.062569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.062852 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.063129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.063991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.066409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.077771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1293/2586] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:51.081805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.092650 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e61a-1b76-4062-aaeb-f618d726c6e5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.097745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f1f17-d8da-4405-b4d9-723d543c07a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.098703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f1f17-d8da-4405-b4d9-723d543c07a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.099005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f1f17-d8da-4405-b4d9-723d543c07a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.099434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f1f17-d8da-4405-b4d9-723d543c07a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.100024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f1f17-d8da-4405-b4d9-723d543c07a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.101887 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f1f17-d8da-4405-b4d9-723d543c07a4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.114825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1294/2587] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:51.119049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.130667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaecbb-dbde-42c1-9caa-84330862662a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.138015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.138809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.138975 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.139325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] PATH_INFO: `/v3/projects/a3db8d0049614b62a0d0d8838d4fbc22/users/c8e0e3fb02d646beae575174cb55243a/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.148458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:51.159371 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3db8d0049614b62a0d0d8838d4fbc22, user_id=c8e0e3fb02d646beae575174cb55243a, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.162943 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.179604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1294/2588] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:51 2022] PUT /identity/v3/projects/a3db8d0049614b62a0d0d8838d4fbc22/users/c8e0e3fb02d646beae575174cb55243a/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:51.191431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.192083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.193005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.238713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] MFA Rules not processed for user `c8e0e3fb02d646beae575174cb55243a`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:51.239980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:51.286913 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:51.313326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1295/2589] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:51 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:51.317876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.389245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4613c1-6c94-4984-82d9-7762945353f3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.393981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.394593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.394872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.395145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.395801 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.397735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.431594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1295/2590] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:51 2022] POST /identity/v3/projects => generated 407 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:51.433844 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.444630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d7212-dfb6-45b3-bb7c-d6285b2cb14c None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.502878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.503493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.503780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.504049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.504700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.506644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.530458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1296/2591] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:51 2022] POST /identity/v3/users => generated 310 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:51.534807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.546276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a446a-7b56-4d19-ac0c-f78dde4e2207 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.551135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.551729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.552012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.552287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.553019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.554622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.565397 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1296/2592] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:51.569796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.581173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f6171-602b-4a1a-a630-73c594efae06 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.586805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.588172 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.588457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.588759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] PATH_INFO: `/v3/projects/84c7ace87880434cbb8d876c4ad04719/users/1e69d0d55ee74943b42fe87e186be2d8/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.606633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c7ace87880434cbb8d876c4ad04719, user_id=1e69d0d55ee74943b42fe87e186be2d8, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.609383 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.623093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1297/2593] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:51 2022] PUT /identity/v3/projects/84c7ace87880434cbb8d876c4ad04719/users/1e69d0d55ee74943b42fe87e186be2d8/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:20:51.627370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.638370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290179e-ab3d-4e38-b10c-1be042c55736 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.688922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.692687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:51.707795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.708752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.709221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.709712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.710421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.712675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.727046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1297/2594] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:51.728902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.739089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376e9a-9aa2-41a2-86f3-6a2e5f45b72c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.744686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d619bfa-2899-4870-8d9c-48ecc207a9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.745396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619bfa-2899-4870-8d9c-48ecc207a9d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.745832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619bfa-2899-4870-8d9c-48ecc207a9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.746486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619bfa-2899-4870-8d9c-48ecc207a9d5 admin admin] PATH_INFO: `/v3/projects/84c7ace87880434cbb8d876c4ad04719/users/1e69d0d55ee74943b42fe87e186be2d8/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.748267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619bfa-2899-4870-8d9c-48ecc207a9d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c7ace87880434cbb8d876c4ad04719, user_id=1e69d0d55ee74943b42fe87e186be2d8, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.751198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619bfa-2899-4870-8d9c-48ecc207a9d5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.763570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1298/2595] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:51 2022] PUT /identity/v3/projects/84c7ace87880434cbb8d876c4ad04719/users/1e69d0d55ee74943b42fe87e186be2d8/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:51.773613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab9c16-11f3-4a00-9245-904d888bc3fb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:51.775117 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.775546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.775830 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.805324 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] MFA Rules not processed for user `1e69d0d55ee74943b42fe87e186be2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:51.807153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:51.828006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:51.839087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:51.839755 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.840040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.840321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] PATH_INFO: `/v3/users/8af9214eabc743c28bb3b8555ac70596` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.841555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af9214eabc743c28bb3b8555ac70596)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:51.843482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:51.845604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:51.890135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1299/2596] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:51 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:51.892982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af9214eabc743c28bb3b8555ac70596'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:51.897864 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:51.899850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:51.901951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af9214eabc743c28bb3b8555ac70596'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:51.909548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af9214eabc743c28bb3b8555ac70596'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:51.910182 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af9214eabc743c28bb3b8555ac70596'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:51.910803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a5ef6937-3c7c-49ee-81ca-3465683cb554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af9214eabc743c28bb3b8555ac70596'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:51.928415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1298/2597] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:51 2022] DELETE /identity/v3/users/8af9214eabc743c28bb3b8555ac70596 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:51.932890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:51.933210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:51.933486 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:51.952121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:51.953352 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:51.959922 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819221ff-6782-445a-ac84-cd80f63e7553 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:52.010172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:52.010809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.011089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.011362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.013249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.013368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1299/2598] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:52.022744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:52.024087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.080969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:52.082458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:52.082858 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f151a72-7b22-4d5a-af39-aeec938b8c09 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:52.086526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:52.087671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.087998 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.088211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.093034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1300/2599] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:51 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:20:52.098010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:52.099443 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:52.110490 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1300/2600] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:20:52.254070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:52.255515 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.265441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:52.266148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.272096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c721091-5f9d-402a-ad9e-31942763d376 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:52.278669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:52.279327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.279610 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.279886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] PATH_INFO: `/v3/users/35eee5f7bf84433793792f9d70b28e18` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.281135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35eee5f7bf84433793792f9d70b28e18)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:52.281331 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39606d-49ef-4b42-8aa2-bf04f873e1ee service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:52.283234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:52.285278 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:52.286242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.286524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.286950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.288807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.307193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:52.333134 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:52.359784 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35eee5f7bf84433793792f9d70b28e18'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.360575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35eee5f7bf84433793792f9d70b28e18'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.372743 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:52.374500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162ac37-4807-4420-a851-97a6a51dae4b service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:52.375920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35eee5f7bf84433793792f9d70b28e18'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.385065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1301/2601] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:52 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:52.387139 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35eee5f7bf84433793792f9d70b28e18'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.400986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35eee5f7bf84433793792f9d70b28e18'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.412192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1301/2602] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:52 2022] DELETE /identity/v3/users/35eee5f7bf84433793792f9d70b28e18 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:52.417426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.417721 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.417994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.423599 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:52.425105 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.472695 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:52.477082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.519192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81352f3e-9263-4b3f-9a89-d61534dc85a5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:52.527566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:52.528730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.529163 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.529565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] PATH_INFO: `/v3/projects/6abea21d51da414d804056066647a6fa` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.531321 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6abea21d51da414d804056066647a6fa)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:52.534249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:52.570784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1302/2603] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:52.581699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:52.583132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.602021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6abea21d51da414d804056066647a6fa', 'request_id': 'req-915d3251-1db5-44aa-a8c8-96c8b244de2b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.615338 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6abea21d51da414d804056066647a6fa', 'request_id': 'req-915d3251-1db5-44aa-a8c8-96c8b244de2b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.625266 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6abea21d51da414d804056066647a6fa', 'request_id': 'req-915d3251-1db5-44aa-a8c8-96c8b244de2b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.647247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] The token cache is being invalidate because project 6abea21d51da414d804056066647a6fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:52.647607 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.648420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:52.650127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1302/2604] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:52 2022] DELETE /identity/v3/projects/6abea21d51da414d804056066647a6fa => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:52.673932 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87416b-6bf2-4915-bf2e-e9b2579a696f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:52.744445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:52.745362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:52.745765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:52.746151 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:52.749280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:52.786250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:52.878643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:52.880617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:52.904592 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1303/2605] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:20:53.317367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:53.320775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:53.341577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d3251-1db5-44aa-a8c8-96c8b244de2b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:53.355582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:53.356485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:53.356919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:53.357224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] PATH_INFO: `/v3/projects/02ea788020be400ea141e8abeb59f6ac` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:53.360231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ea788020be400ea141e8abeb59f6ac)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:53.362565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:53.409465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:53.428845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ea788020be400ea141e8abeb59f6ac', 'request_id': 'req-06a3a29a-3a6e-40fe-a184-cffee02572ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.440611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ea788020be400ea141e8abeb59f6ac', 'request_id': 'req-06a3a29a-3a6e-40fe-a184-cffee02572ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.450049 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ea788020be400ea141e8abeb59f6ac', 'request_id': 'req-06a3a29a-3a6e-40fe-a184-cffee02572ee'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.466047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] The token cache is being invalidate because project 02ea788020be400ea141e8abeb59f6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:53.466437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.467734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.469776 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1303/2606] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:53 2022] DELETE /identity/v3/projects/02ea788020be400ea141e8abeb59f6ac => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:53.473622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:53.486616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7acb27-47c0-404f-8e11-f3a3ac8c4da7 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:53.569719 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:53.570388 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:53.570667 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:53.570942 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] PATH_INFO: `/v3/users/3ed4f5ed8f544434a32b1bfcfaa898a6` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:53.576224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed4f5ed8f544434a32b1bfcfaa898a6)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:53.578700 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:53.654217 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed4f5ed8f544434a32b1bfcfaa898a6'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.663652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed4f5ed8f544434a32b1bfcfaa898a6'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.692366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed4f5ed8f544434a32b1bfcfaa898a6'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.693097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed4f5ed8f544434a32b1bfcfaa898a6'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.693800 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1906196f-944b-4acd-821f-a8742f5e407e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed4f5ed8f544434a32b1bfcfaa898a6'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:53.719524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1304/2607] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:53 2022] DELETE /identity/v3/users/3ed4f5ed8f544434a32b1bfcfaa898a6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:54.247506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.249285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:54.271167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3a29a-3a6e-40fe-a184-cffee02572ee admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.342639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.343364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.343714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.344060 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] PATH_INFO: `/v3/projects/153417d829c24d58bdbe9f04e67670fa` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.345366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=153417d829c24d58bdbe9f04e67670fa)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.347320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.397428 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153417d829c24d58bdbe9f04e67670fa', 'request_id': 'req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:54.409977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153417d829c24d58bdbe9f04e67670fa', 'request_id': 'req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:54.417269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153417d829c24d58bdbe9f04e67670fa', 'request_id': 'req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:54.427613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] The token cache is being invalidate because project 153417d829c24d58bdbe9f04e67670fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:54.427897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:54.428552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-adfc42c7-fe6c-49aa-bb76-a2df6c08a3c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:54.430106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1304/2608] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:54 2022] DELETE /identity/v3/projects/153417d829c24d58bdbe9f04e67670fa => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:54.436417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.436909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.437136 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.458622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:54.459975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:54.504132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:54.527684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1305/2609] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:54.536983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.537290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.537567 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.558335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:54.559539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:54.567542 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1305/2610] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:54.572366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.582287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e037-5a31-49bf-a36a-488875a8d8d8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.587549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.588171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.588447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.588713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.589434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:54.589761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.591336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.602827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1306/2611] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:20:54.607718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.617072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bdc3a-b6f9-4442-bd15-5f160b512d04 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.622028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.622637 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.622918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.623194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.624135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.626095 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.654014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1306/2612] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:54 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:54.658112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.668032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf09884-b3d9-4a67-9d1a-89ac7fcefc79 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.724475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.725135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.725423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.725699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.726363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.729065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.751514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1307/2613] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:54 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:54.755649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.766671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a17f92-bfa2-48df-9798-589053f538e7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.771387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.771998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.772279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.772554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.773289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.774996 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.785070 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1307/2614] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:54.789041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.799917 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae720051-e6e8-47eb-b318-60ddd518d6d5 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.804576 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f214a81-5e10-47c2-92a0-09922ad2bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.805208 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f214a81-5e10-47c2-92a0-09922ad2bbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.806156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f214a81-5e10-47c2-92a0-09922ad2bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.806437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f214a81-5e10-47c2-92a0-09922ad2bbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.807127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f214a81-5e10-47c2-92a0-09922ad2bbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.808725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f214a81-5e10-47c2-92a0-09922ad2bbe7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.819321 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1308/2615] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:54.824025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:54.834005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693546c-a7a0-46a4-a78f-cfa7eefd06ba admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:54.839136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:54.839819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.840094 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.840366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] PATH_INFO: `/v3/projects/0660e286031645e49b2fd76dc9cf2de0/users/6a47d06754304dab9ea939af8dfe105a/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.848689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:54.856674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0660e286031645e49b2fd76dc9cf2de0, user_id=6a47d06754304dab9ea939af8dfe105a, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:54.860004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:54.874021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1308/2616] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:54 2022] PUT /identity/v3/projects/0660e286031645e49b2fd76dc9cf2de0/users/6a47d06754304dab9ea939af8dfe105a/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:54.885555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:54.885834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:54.886171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:54.917196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] MFA Rules not processed for user `6a47d06754304dab9ea939af8dfe105a`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:54.918325 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:54.986509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1309/2617] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:20:54.998657 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:55.000451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:55.062077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c1688-e08c-4406-8907-d820232a5299 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:55.065958 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2555-c012-4468-9735-03dee3b677e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:55.066682 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2555-c012-4468-9735-03dee3b677e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.066963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2555-c012-4468-9735-03dee3b677e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.067473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2555-c012-4468-9735-03dee3b677e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.077044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2555-c012-4468-9735-03dee3b677e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:55.078477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2555-c012-4468-9735-03dee3b677e1 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:55.088405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1309/2618] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:55.110316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:55.152770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:55.176433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c53ff8-5607-4678-8a89-6e72939067d8 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:55.181154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:55.181851 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.182131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.182409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] PATH_INFO: `/v3/users/6a47d06754304dab9ea939af8dfe105a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.183659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a47d06754304dab9ea939af8dfe105a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:55.185619 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:55.231129 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a47d06754304dab9ea939af8dfe105a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.237974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a47d06754304dab9ea939af8dfe105a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.244753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a47d06754304dab9ea939af8dfe105a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.245405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a47d06754304dab9ea939af8dfe105a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.246024 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a47d06754304dab9ea939af8dfe105a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.264763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1310/2619] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:55 2022] DELETE /identity/v3/users/6a47d06754304dab9ea939af8dfe105a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:55.269452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.269949 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.270056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.290083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:55.291349 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:55.355378 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1310/2620] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:55.363769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:55.364992 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:55.427644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9bcfd-15a0-48bb-afeb-7cd13e6f94cb admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:55.431661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a620f-1e20-41b5-8d9d-c5a8007106ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:55.432302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a620f-1e20-41b5-8d9d-c5a8007106ad service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.432583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a620f-1e20-41b5-8d9d-c5a8007106ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.432918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a620f-1e20-41b5-8d9d-c5a8007106ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.443733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a620f-1e20-41b5-8d9d-c5a8007106ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:55.445216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a620f-1e20-41b5-8d9d-c5a8007106ad service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:55.456319 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1311/2621] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:55.728323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:55.729963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:55.745782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82742978-0b00-4028-b019-4d37ddd6013a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:55.750646 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:55.751293 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.751573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.751938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] PATH_INFO: `/v3/projects/0660e286031645e49b2fd76dc9cf2de0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.753014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0660e286031645e49b2fd76dc9cf2de0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:55.754915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:55.808151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0660e286031645e49b2fd76dc9cf2de0', 'request_id': 'req-86e4bf5c-c883-405d-8b88-45bef93c2f51'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.815950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0660e286031645e49b2fd76dc9cf2de0', 'request_id': 'req-86e4bf5c-c883-405d-8b88-45bef93c2f51'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.824084 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0660e286031645e49b2fd76dc9cf2de0', 'request_id': 'req-86e4bf5c-c883-405d-8b88-45bef93c2f51'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.834897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] The token cache is being invalidate because project 0660e286031645e49b2fd76dc9cf2de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:55.835201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.835845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-86e4bf5c-c883-405d-8b88-45bef93c2f51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:55.837696 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1311/2622] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:55 2022] DELETE /identity/v3/projects/0660e286031645e49b2fd76dc9cf2de0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:55.842647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.842933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.843283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.862433 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:55.863620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:55.914417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:55.924345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1312/2623] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:55.932604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.932898 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.933216 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.954227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:55.955442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:55.963493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1312/2624] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:55.967633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:55.977891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b931a7-09fa-4443-a1a4-7bccab4b7992 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:55.982340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:55.982951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:55.983223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:55.983492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:55.984177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:55.984476 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:55.986091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:55.999170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1313/2625] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:20:56.003516 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.012951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e42e64-3b54-41e6-8cb4-4900921a1c38 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.017697 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.018303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.018585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.018859 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.019530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.021469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.026097 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:56.051233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1313/2626] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:56 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:56.056364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.066338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981371b9-7d38-4eb1-9199-b98f05e2cf2f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.127984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.128709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.129023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.129302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.129967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.131918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.164542 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1314/2627] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:56 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:20:56.165979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.176669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492683-dc33-4538-9510-0de614e771bd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.182318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.183093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.183222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.183498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.184206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.185915 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.200423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1314/2628] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:20:56.202441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.214007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffbe84-cad4-44d8-a58b-feb24adbc0ff admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.218736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4816a3-0211-4cd0-aeae-53b9074ec5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.219400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4816a3-0211-4cd0-aeae-53b9074ec5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.219789 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4816a3-0211-4cd0-aeae-53b9074ec5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.219959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4816a3-0211-4cd0-aeae-53b9074ec5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.220663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4816a3-0211-4cd0-aeae-53b9074ec5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.222475 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4816a3-0211-4cd0-aeae-53b9074ec5aa admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.234554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1315/2629] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:56.238400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.248687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f0f95-9eee-4445-956c-a4730e86927a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.256102 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.256775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.257082 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.257453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] PATH_INFO: `/v3/projects/4b5f0efb37434a488a57b0baeb884da1/users/7b3065b8d05e4e3086b4a840779213a7/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.277612 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5f0efb37434a488a57b0baeb884da1, user_id=7b3065b8d05e4e3086b4a840779213a7, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.280302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.293020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1315/2630] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:56 2022] PUT /identity/v3/projects/4b5f0efb37434a488a57b0baeb884da1/users/7b3065b8d05e4e3086b4a840779213a7/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:20:56.303795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.304090 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.304371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.335456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] MFA Rules not processed for user `7b3065b8d05e4e3086b4a840779213a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:56.337367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:56.402229 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1316/2631] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:56.436126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.437867 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:56.495786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930114f-2a95-4577-96cf-7b7a169f51cd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.499534 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0fde3-0f10-4caf-8fd2-e03f384ccd81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.500144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0fde3-0f10-4caf-8fd2-e03f384ccd81 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.500421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0fde3-0f10-4caf-8fd2-e03f384ccd81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.500693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0fde3-0f10-4caf-8fd2-e03f384ccd81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.509627 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0fde3-0f10-4caf-8fd2-e03f384ccd81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.511065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0fde3-0f10-4caf-8fd2-e03f384ccd81 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.519875 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1316/2632] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:56.535275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.597176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:56.605210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50418e78-66b3-4891-a0cd-a93c4b28318f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.611142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.611961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.612355 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.612657 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] PATH_INFO: `/v3/users/7b3065b8d05e4e3086b4a840779213a7` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.613944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3065b8d05e4e3086b4a840779213a7)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.615999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.666652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3065b8d05e4e3086b4a840779213a7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:56.673537 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3065b8d05e4e3086b4a840779213a7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:56.681508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3065b8d05e4e3086b4a840779213a7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:56.682221 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3065b8d05e4e3086b4a840779213a7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:56.683059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3065b8d05e4e3086b4a840779213a7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:56.708636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1317/2633] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:20:56 2022] DELETE /identity/v3/users/7b3065b8d05e4e3086b4a840779213a7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:56.714023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.714308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.714581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.734416 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:56.735546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:56.796259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1317/2634] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:56.806381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:56.807883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:56.863914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2061e5-9c43-4aa8-825f-69b17fbf1f1b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:56.867489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71f6d0-3919-484c-a01e-0ced732d1f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:56.868747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71f6d0-3919-484c-a01e-0ced732d1f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:56.869051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71f6d0-3919-484c-a01e-0ced732d1f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:56.869326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71f6d0-3919-484c-a01e-0ced732d1f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:56.878456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71f6d0-3919-484c-a01e-0ced732d1f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:56.879883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71f6d0-3919-484c-a01e-0ced732d1f87 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:56.889575 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1318/2635] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:20:57.155925 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.158103 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:57.173468 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b7408-4305-4f1e-b78c-6e05e460319b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.178411 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.179062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.179359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.179669 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] PATH_INFO: `/v3/projects/4b5f0efb37434a488a57b0baeb884da1` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.181525 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5f0efb37434a488a57b0baeb884da1)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.183372 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.239015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5f0efb37434a488a57b0baeb884da1', 'request_id': 'req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:57.244109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:57.251625 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5f0efb37434a488a57b0baeb884da1', 'request_id': 'req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:57.263343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5f0efb37434a488a57b0baeb884da1', 'request_id': 'req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:57.277554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] The token cache is being invalidate because project 4b5f0efb37434a488a57b0baeb884da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:20:57.277843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:57.278522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-be553ae2-c9f8-4f1b-8520-afb1cfb9a33b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:20:57.280092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1318/2636] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:20:57 2022] DELETE /identity/v3/projects/4b5f0efb37434a488a57b0baeb884da1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:57.285070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.285362 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.285644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.308417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:57.310196 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:57.404548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1319/2637] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:57.412106 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.412394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.412674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.434924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:57.436346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:57.449993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1319/2638] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:57.451180 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.461686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e88b4-f30a-43ec-b2f8-1c5e864fe048 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.471149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.471149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.471149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.471149 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.472021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:20:57.472021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.474551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.490267 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1320/2639] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:20:57.502426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.512046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1032e51d-da7d-42d0-9ea1-a547ac98abdd None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.523242 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.524376 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.524857 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.525469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.526328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.529939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.573342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1320/2640] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:20:57 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:57.576661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.583899 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:20:57.592010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfabccf-e54b-4c61-a85b-3bf1140dc69e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.675988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.677156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.677156 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.678358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.678529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.682080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.722241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1321/2641] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:20:57 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:20:57.723458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.745825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1454445-9c3c-40ec-beda-2fb9f960e67e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.790485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.792320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.792854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.793391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.794499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.796520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.817735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1321/2642] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:20:57.818131 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.833166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdcc23-371c-434d-9548-e706f2d2dfe3 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.839413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176fd51-aa5c-4507-8087-9eb03c1b2064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.839973 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176fd51-aa5c-4507-8087-9eb03c1b2064 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.840265 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176fd51-aa5c-4507-8087-9eb03c1b2064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.840524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176fd51-aa5c-4507-8087-9eb03c1b2064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.841263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176fd51-aa5c-4507-8087-9eb03c1b2064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.843338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176fd51-aa5c-4507-8087-9eb03c1b2064 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.856563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1322/2643] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:20:57.861211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:57.874313 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227c27d-ead3-4454-bfed-351f5bb30ec5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:57.880420 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:57.881207 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.881568 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.881807 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] PATH_INFO: `/v3/projects/461f4d3a0a984e258de729b40739e47d/users/68a610aeb01e4fc48871c8143c2e4c31/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.904992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=461f4d3a0a984e258de729b40739e47d, user_id=68a610aeb01e4fc48871c8143c2e4c31, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:57.907894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:57.924550 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1322/2644] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:20:57 2022] PUT /identity/v3/projects/461f4d3a0a984e258de729b40739e47d/users/68a610aeb01e4fc48871c8143c2e4c31/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:20:57.935491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:57.935843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:57.936134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:57.968369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] MFA Rules not processed for user `68a610aeb01e4fc48871c8143c2e4c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:20:57.969723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:58.040854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1323/2645] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:20:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:20:58.053451 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:20:58.055377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:20:58.114051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d90438-fcc9-4739-b3c4-27ed60f4b6fd admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:20:58.117651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f34cd-1fa7-491a-a223-6fa2ebe0c299 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:20:58.118235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f34cd-1fa7-491a-a223-6fa2ebe0c299 service glance] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:20:58.118518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f34cd-1fa7-491a-a223-6fa2ebe0c299 service glance] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:20:58.118791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f34cd-1fa7-491a-a223-6fa2ebe0c299 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:20:58.128012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f34cd-1fa7-491a-a223-6fa2ebe0c299 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:20:58.129522 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f34cd-1fa7-491a-a223-6fa2ebe0c299 service glance] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:20:58.138422 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1323/2646] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:20:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:00.148648 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:00.150210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:00.228127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e61e0-4bc4-4bda-a2ea-e6191be812b9 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:00.232683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:00.233361 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:00.233641 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:00.233916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] PATH_INFO: `/v3/users/c8e0e3fb02d646beae575174cb55243a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:00.235109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e0e3fb02d646beae575174cb55243a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:00.237069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:00.252857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:00.292552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e0e3fb02d646beae575174cb55243a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:00.300554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e0e3fb02d646beae575174cb55243a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:00.309664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e0e3fb02d646beae575174cb55243a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:00.310218 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e0e3fb02d646beae575174cb55243a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:00.310847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e0e3fb02d646beae575174cb55243a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:00.330189 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1324/2647] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:00 2022] DELETE /identity/v3/users/c8e0e3fb02d646beae575174cb55243a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:00.334767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:00.335243 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:00.335418 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:00.356363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:00.357620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:00.419540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1324/2648] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:00.429206 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:00.431213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:00.491678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60477f-4600-4c3a-b137-fcdebe89001c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:00.496323 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:00.496947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:00.497252 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:00.497523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:00.510167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:00.511684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:00.521871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1325/2649] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:00.800019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:00.801562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:00.814620 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ce659-1974-4be5-a18c-46518bfdede6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:00.819309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:00.820036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:00.820379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:00.820723 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] PATH_INFO: `/v3/projects/a3db8d0049614b62a0d0d8838d4fbc22` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:00.822486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3db8d0049614b62a0d0d8838d4fbc22)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:00.824379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:00.835900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:01.004309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3db8d0049614b62a0d0d8838d4fbc22', 'request_id': 'req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.011790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3db8d0049614b62a0d0d8838d4fbc22', 'request_id': 'req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.018747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3db8d0049614b62a0d0d8838d4fbc22', 'request_id': 'req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.034666 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] The token cache is being invalidate because project a3db8d0049614b62a0d0d8838d4fbc22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:01.034969 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.035636 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.037475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1325/2650] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:00 2022] DELETE /identity/v3/projects/a3db8d0049614b62a0d0d8838d4fbc22 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:01.046127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:01.055953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f391c88-d37f-47da-ab4d-b01c09a677ea service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:01.116555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:01.117228 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.117512 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.117819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] PATH_INFO: `/v3/users/1e69d0d55ee74943b42fe87e186be2d8` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.119216 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e69d0d55ee74943b42fe87e186be2d8)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:01.121167 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:01.165276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:01.175987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e69d0d55ee74943b42fe87e186be2d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.188547 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e69d0d55ee74943b42fe87e186be2d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.198310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e69d0d55ee74943b42fe87e186be2d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.199002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e69d0d55ee74943b42fe87e186be2d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.199726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-872deda3-4c96-4cb5-9c3d-113be1cf5c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e69d0d55ee74943b42fe87e186be2d8'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.219170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1326/2651] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:01 2022] DELETE /identity/v3/users/1e69d0d55ee74943b42fe87e186be2d8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:01.514111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:01.515887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:01.532033 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5df36-76e6-4dea-9cc6-e3483fd1fac2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:01.591440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:01.592091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.592370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.592642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] PATH_INFO: `/v3/projects/84c7ace87880434cbb8d876c4ad04719` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.595229 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c7ace87880434cbb8d876c4ad04719)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:01.598081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:01.655762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c7ace87880434cbb8d876c4ad04719', 'request_id': 'req-14b50306-0cd7-4e1e-8136-b999852f087c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.662654 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c7ace87880434cbb8d876c4ad04719', 'request_id': 'req-14b50306-0cd7-4e1e-8136-b999852f087c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.672917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c7ace87880434cbb8d876c4ad04719', 'request_id': 'req-14b50306-0cd7-4e1e-8136-b999852f087c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.688739 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] The token cache is being invalidate because project 84c7ace87880434cbb8d876c4ad04719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:01.689064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.689727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-14b50306-0cd7-4e1e-8136-b999852f087c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:01.691238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1326/2652] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:01 2022] DELETE /identity/v3/projects/84c7ace87880434cbb8d876c4ad04719 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:01.696909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.697390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.697677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.717712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:01.718945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:01.786114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1327/2653] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:01.794814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.795092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.795363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.816671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:01.817939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:01.826710 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1327/2654] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:01.831140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:01.841287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99196061-65a1-4b8c-8380-ff25f6f4d4fd None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:01.845952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:01.846504 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.846780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.847049 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.847736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:21:01.848032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:01.849672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:01.864398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1328/2655] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:21:01.869297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:01.878903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f36a-05b6-4b5d-9b5f-c498d4f4ef5b None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:01.884584 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:01.885249 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.885530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.885805 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.886476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:01.888391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:01.913699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:01.921642 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1328/2656] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:01 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:01.925633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:01.936632 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3012f-a263-42c1-b41f-f6288e7788ef admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:01.997224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:01.997878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:01.998164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:01.998707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:01.999420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:02.001428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:02.023509 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1329/2657] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:01 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:21:02.028153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:02.038808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5df87-3a92-44e9-8615-48174587ab41 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:02.044133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:02.044757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:02.045080 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:02.045358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:02.046063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:02.047655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:02.063026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1329/2658] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:02.067150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:02.077437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bc0d7-1b49-4026-abd5-280c26b30607 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:02.083025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fc1ea-5c7a-472c-b8ce-6d13b534b55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:02.083664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fc1ea-5c7a-472c-b8ce-6d13b534b55d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:02.083948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fc1ea-5c7a-472c-b8ce-6d13b534b55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:02.084225 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fc1ea-5c7a-472c-b8ce-6d13b534b55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:02.084928 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fc1ea-5c7a-472c-b8ce-6d13b534b55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:02.087356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fc1ea-5c7a-472c-b8ce-6d13b534b55d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:02.100915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1330/2659] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:02.105580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:02.116158 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68700d11-3250-41fe-8eb6-b700b4555fd8 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:02.122523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:02.123470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:02.123760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:02.124066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] PATH_INFO: `/v3/projects/4a797840d6824b6ebca7654099119541/users/a3c8136d2e1342d396fe2b2d0e67cbfe/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:02.140339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a797840d6824b6ebca7654099119541, user_id=a3c8136d2e1342d396fe2b2d0e67cbfe, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:02.143027 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:02.160387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1330/2660] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:02 2022] PUT /identity/v3/projects/4a797840d6824b6ebca7654099119541/users/a3c8136d2e1342d396fe2b2d0e67cbfe/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:02.172790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:02.173384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:02.173500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:02.205134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] MFA Rules not processed for user `a3c8136d2e1342d396fe2b2d0e67cbfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:02.206398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:02.215564 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:02.282187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1331/2661] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:02.296092 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:02.297845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:02.368317 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ab8a-133a-4680-b959-7cc64fb43e7b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:02.371832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45f269-bdb6-477a-95f9-b8e424026b7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:02.372452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f269-bdb6-477a-95f9-b8e424026b7e service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:02.372726 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f269-bdb6-477a-95f9-b8e424026b7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:02.373023 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f269-bdb6-477a-95f9-b8e424026b7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:02.382164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45f269-bdb6-477a-95f9-b8e424026b7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:02.383667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df45f269-bdb6-477a-95f9-b8e424026b7e service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:02.392851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1331/2662] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:05.643412 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:05.770500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266550c6-49aa-42de-a4f3-813f17ee8f07 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:05.777009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:05.778082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:05.778281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:05.778540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] PATH_INFO: `/v3/users/a3c8136d2e1342d396fe2b2d0e67cbfe` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:05.780559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c8136d2e1342d396fe2b2d0e67cbfe)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:05.782778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:05.855279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c8136d2e1342d396fe2b2d0e67cbfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:05.865318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c8136d2e1342d396fe2b2d0e67cbfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:05.877852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c8136d2e1342d396fe2b2d0e67cbfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:05.878599 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c8136d2e1342d396fe2b2d0e67cbfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:05.879765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c8136d2e1342d396fe2b2d0e67cbfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:05.910209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1332/2663] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:05 2022] DELETE /identity/v3/users/a3c8136d2e1342d396fe2b2d0e67cbfe => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:05.912274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:05.912799 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:05.913771 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:05.942481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:05.943764 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:06.037366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1332/2664] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:06.047900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:06.049117 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:06.086778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:06.124173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20248d2-30cd-4442-9b89-8029f1284bdf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:06.130805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b24fe-c69e-4d4a-8878-905a4b4b7f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:06.132286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b24fe-c69e-4d4a-8878-905a4b4b7f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.132286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b24fe-c69e-4d4a-8878-905a4b4b7f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.132286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b24fe-c69e-4d4a-8878-905a4b4b7f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.144268 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b24fe-c69e-4d4a-8878-905a4b4b7f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:06.146462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b24fe-c69e-4d4a-8878-905a4b4b7f99 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:06.159441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1333/2665] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:06.496312 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:06.497929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:06.513577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112e543-6e9b-420b-bd03-4e5be219872a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:06.519061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:06.519835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.520008 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.520285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] PATH_INFO: `/v3/projects/4a797840d6824b6ebca7654099119541` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.521806 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a797840d6824b6ebca7654099119541)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:06.524663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:06.579253 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a797840d6824b6ebca7654099119541', 'request_id': 'req-455a4556-aa62-4127-835a-0be39856ba0c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:06.589722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a797840d6824b6ebca7654099119541', 'request_id': 'req-455a4556-aa62-4127-835a-0be39856ba0c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:06.599259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a797840d6824b6ebca7654099119541', 'request_id': 'req-455a4556-aa62-4127-835a-0be39856ba0c'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:06.603151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:06.616544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] The token cache is being invalidate because project 4a797840d6824b6ebca7654099119541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:06.617056 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:06.617877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-455a4556-aa62-4127-835a-0be39856ba0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:06.620400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1333/2666] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:06 2022] DELETE /identity/v3/projects/4a797840d6824b6ebca7654099119541 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:06.626397 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.626918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.627193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.655369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:06.656587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:06.730516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1334/2667] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:06.737792 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.738093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.738347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.760244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:06.761537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:06.777178 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1334/2668] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:06.778785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:06.790001 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6f67e-5171-49d8-8146-8cd5e2e9a953 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:06.794975 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:06.795601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.795875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.796146 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.796842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:21:06.797170 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:06.798780 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:06.811595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1335/2669] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:21:06.816408 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:06.827215 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fcd14-89f4-46c3-9f72-266a2cd95e23 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:06.831845 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:06.832458 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.832738 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.833087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.833708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:06.835605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:06.869763 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1335/2670] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:06 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:21:06.874308 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:06.885116 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd54f03-b6fb-446b-9b9f-d904424c7656 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:06.956723 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:06.957375 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:06.957653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:06.957921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:06.958607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:06.960541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:06.984510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1336/2671] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:06 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:06.988910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:07.001938 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ae962-fcd3-4ea6-bc2d-2147f40099ed admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:07.007402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:07.007402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:07.007866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:07.007866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:07.009483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:07.011385 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:07.021829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1336/2672] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:07.026058 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:07.037679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259aed29-70da-4d6c-a947-192584b0b338 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:07.044406 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d5c9b-cdaa-4e2d-912c-6669f842f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:07.045232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d5c9b-cdaa-4e2d-912c-6669f842f82b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:07.045552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d5c9b-cdaa-4e2d-912c-6669f842f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:07.045828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d5c9b-cdaa-4e2d-912c-6669f842f82b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:07.046804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d5c9b-cdaa-4e2d-912c-6669f842f82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:07.048499 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d5c9b-cdaa-4e2d-912c-6669f842f82b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:07.065342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1337/2673] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:21:07.066846 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:07.079162 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a1228-6916-4813-a8ab-cefe1db3f3c0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:07.085902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:07.087021 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:07.087476 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:07.087855 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] PATH_INFO: `/v3/projects/ea7d64052a0d404cba89576286a50bc8/users/f4233f4fc78e4525b55450bc8c673e8a/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:07.112192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7d64052a0d404cba89576286a50bc8, user_id=f4233f4fc78e4525b55450bc8c673e8a, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:07.115068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:07.134693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1337/2674] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:07 2022] PUT /identity/v3/projects/ea7d64052a0d404cba89576286a50bc8/users/f4233f4fc78e4525b55450bc8c673e8a/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:07.145740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:07.146390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:07.146712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:07.189176 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] MFA Rules not processed for user `f4233f4fc78e4525b55450bc8c673e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:07.190470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:07.238132 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:07.287880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1338/2675] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:07.302545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:07.304667 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:07.377746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c05f40-379d-41d0-8ea0-3c930aa28188 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:07.381910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff190da-fc94-4981-8ab0-1352ea3872a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:07.382916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff190da-fc94-4981-8ab0-1352ea3872a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:07.383448 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff190da-fc94-4981-8ab0-1352ea3872a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:07.384524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff190da-fc94-4981-8ab0-1352ea3872a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:07.395917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff190da-fc94-4981-8ab0-1352ea3872a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:07.397487 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff190da-fc94-4981-8ab0-1352ea3872a5 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:07.407818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1338/2676] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:21:15.779707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:15.902665 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8aecc-2d6f-4d87-ac18-de0beaf37cb6 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:15.908327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:15.909094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:15.909386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:15.909712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] PATH_INFO: `/v3/users/f4233f4fc78e4525b55450bc8c673e8a` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:15.911577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4233f4fc78e4525b55450bc8c673e8a)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:15.914039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:16.015199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4233f4fc78e4525b55450bc8c673e8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.032094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4233f4fc78e4525b55450bc8c673e8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.046227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4233f4fc78e4525b55450bc8c673e8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.047113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4233f4fc78e4525b55450bc8c673e8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.047971 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4233f4fc78e4525b55450bc8c673e8a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.088624 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1339/2677] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:15 2022] DELETE /identity/v3/users/f4233f4fc78e4525b55450bc8c673e8a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:16.093587 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65487b35-c524-474e-91ec-b40150494c6d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:16.094331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65487b35-c524-474e-91ec-b40150494c6d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:16.094331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65487b35-c524-474e-91ec-b40150494c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:16.120889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-65487b35-c524-474e-91ec-b40150494c6d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:16.121521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-65487b35-c524-474e-91ec-b40150494c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:16.229469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1339/2678] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:16.236784 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:16.238366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:16.307427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:16.315043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94768ac-bab8-4160-b308-464f8c3fc7e7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:16.323972 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:16.324663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:16.325341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:16.325693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:16.336305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:16.338570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:16.357227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1340/2679] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:16.418423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65487b35-c524-474e-91ec-b40150494c6d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:16.420459 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-65487b35-c524-474e-91ec-b40150494c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:16.473437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:16.567263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65487b35-c524-474e-91ec-b40150494c6d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:16.581332 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:16.582130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:16.582331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:16.582640 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:16.595194 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:16.619663 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:16.621554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1beb8-dfdc-40d4-9196-cf69bd4acf28 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:16.635927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1340/2680] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:16.855770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:16.857336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:16.872678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c2c50-ddda-4b89-a6a0-d2f53cf2c583 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:16.879145 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:16.879649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:16.880211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:16.880578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] PATH_INFO: `/v3/projects/ea7d64052a0d404cba89576286a50bc8` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:16.881959 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7d64052a0d404cba89576286a50bc8)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:16.884460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:16.961398 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7d64052a0d404cba89576286a50bc8', 'request_id': 'req-1598b322-7f89-47c7-b2b5-b58bea9e66d7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.969752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7d64052a0d404cba89576286a50bc8', 'request_id': 'req-1598b322-7f89-47c7-b2b5-b58bea9e66d7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.977847 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7d64052a0d404cba89576286a50bc8', 'request_id': 'req-1598b322-7f89-47c7-b2b5-b58bea9e66d7'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.995403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] The token cache is being invalidate because project ea7d64052a0d404cba89576286a50bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:16.995735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.996629 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1598b322-7f89-47c7-b2b5-b58bea9e66d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:16.998863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1341/2681] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:16 2022] DELETE /identity/v3/projects/ea7d64052a0d404cba89576286a50bc8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:17.003744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.004047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.004649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.032007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:17.033546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:17.105197 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1341/2682] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:17.114596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6979-c916-4d96-aed9-af845e584eec None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.114901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6979-c916-4d96-aed9-af845e584eec None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.115153 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6979-c916-4d96-aed9-af845e584eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.138439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-346e6979-c916-4d96-aed9-af845e584eec None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:17.139689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-346e6979-c916-4d96-aed9-af845e584eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:17.148816 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1342/2683] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:17.154457 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.173605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ab4af-7b9b-431d-8c86-90cf3a48e6d8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.181035 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.181755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.182081 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.182370 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.183404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:21:17.183962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.187156 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.208063 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1342/2684] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:21:17.213646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6979-c916-4d96-aed9-af845e584eec None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.224622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6979-c916-4d96-aed9-af845e584eec None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.231764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.232924 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.233524 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.234023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.235307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.238095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.273248 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1343/2685] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:17 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:17.277899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.291920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd013fdc-53a7-46d6-b6e0-7b87ba5e0845 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.371823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.372409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.372688 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.373233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.373941 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.375984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.405062 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1343/2686] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:17 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:21:17.406778 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.420057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b2eff-b28d-46fb-b291-b79413cc28de admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.425415 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.427016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.427016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.427016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.428133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.430521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.445802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1344/2687] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:17.447257 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.459549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a217c5-93e7-49cd-bc6c-e054c82bc92b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.465877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc76cf-5d19-4236-bea7-27381552bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.466520 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc76cf-5d19-4236-bea7-27381552bb20 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.466856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc76cf-5d19-4236-bea7-27381552bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.467075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc76cf-5d19-4236-bea7-27381552bb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.467772 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc76cf-5d19-4236-bea7-27381552bb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.469442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc76cf-5d19-4236-bea7-27381552bb20 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.481918 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1344/2688] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:17.486442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.506317 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874e053-6a5e-44ee-a37e-673a6a552915 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.515565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.516473 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.516903 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.517366 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] PATH_INFO: `/v3/projects/6bc7b7b99e854517a3a43d0b7f7b471a/users/154a6b37687d471da649a8f6727d751e/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.546908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bc7b7b99e854517a3a43d0b7f7b471a, user_id=154a6b37687d471da649a8f6727d751e, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.552724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.578036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1345/2689] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:17 2022] PUT /identity/v3/projects/6bc7b7b99e854517a3a43d0b7f7b471a/users/154a6b37687d471da649a8f6727d751e/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:17.586191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196124ad-3831-4d78-a77c-a3f0a3ac566a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.587679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196124ad-3831-4d78-a77c-a3f0a3ac566a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.588157 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196124ad-3831-4d78-a77c-a3f0a3ac566a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.612083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:17.624125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-196124ad-3831-4d78-a77c-a3f0a3ac566a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:17.626660 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-196124ad-3831-4d78-a77c-a3f0a3ac566a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:17.736005 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1345/2690] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:17.746877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:17.748982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:17.833812 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39410ef9-0bf5-4c3b-aeb8-feb3b7285694 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:17.838187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:17.838831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:17.839074 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:17.839354 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:17.852114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:17.855407 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:17.865849 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1346/2691] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:22.856693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:22.857160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:22.857259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:22.904630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] MFA Rules not processed for user `154a6b37687d471da649a8f6727d751e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:22.905976 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:22.983826 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1346/2692] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:22.987420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:22.998843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee97c-9efd-435e-9608-98d42f39124f service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:23.004994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:23.005675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:23.005960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:23.006234 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:23.006912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:23.010647 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:23.025718 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:23.056213 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1347/2693] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:22 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:23.057493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:23.069662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa670e-e859-4f63-9fbd-95ab2422e7dc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:23.147014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:23.147733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:23.148987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:23.148987 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:23.152538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:23.154549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:23.181012 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1347/2694] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:23 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:23.184127 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:23.203294 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3aface-3902-4141-9969-76a94afe4c5b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:23.209909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:23.211656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:23.212057 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:23.212403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:23.213236 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:23.215492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:23.226825 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1348/2695] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:23.231882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:23.245046 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c9406-e5dc-43f5-996f-f23d280e4ca9 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:23.250326 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:23.251018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:23.251296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:23.251591 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] PATH_INFO: `/v3/projects/2feec93864504689a1d5973c9f747bf4/users/5ba9373ffcab4af6bdda962a51c3e6c9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:23.270308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feec93864504689a1d5973c9f747bf4, user_id=5ba9373ffcab4af6bdda962a51c3e6c9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:23.274154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:23.289499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1348/2696] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:23 2022] PUT /identity/v3/projects/2feec93864504689a1d5973c9f747bf4/users/5ba9373ffcab4af6bdda962a51c3e6c9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:21:23.295123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:23.306119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2693ebf-3a2f-416f-88d1-a92e1ff5c07e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:23.417631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:23.419009 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:23.419457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:23.419838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:23.420637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:23.422633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:23.434340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1349/2697] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:23.445007 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:23.452691 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:23.460235 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bcb6d-8bca-4b45-a69b-26c253b6c1bf admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:23.467151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271acb9-bb0d-41f0-8e8a-5c0886ca8dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:23.467866 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271acb9-bb0d-41f0-8e8a-5c0886ca8dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:23.468622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271acb9-bb0d-41f0-8e8a-5c0886ca8dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:23.468622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271acb9-bb0d-41f0-8e8a-5c0886ca8dde admin admin] PATH_INFO: `/v3/projects/2feec93864504689a1d5973c9f747bf4/users/5ba9373ffcab4af6bdda962a51c3e6c9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:23.470780 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271acb9-bb0d-41f0-8e8a-5c0886ca8dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=2feec93864504689a1d5973c9f747bf4, user_id=5ba9373ffcab4af6bdda962a51c3e6c9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:23.473527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271acb9-bb0d-41f0-8e8a-5c0886ca8dde admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:23.487910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1349/2698] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:23 2022] PUT /identity/v3/projects/2feec93864504689a1d5973c9f747bf4/users/5ba9373ffcab4af6bdda962a51c3e6c9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:24.334448 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:24.474292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da4108-3c7e-470a-a2b6-82fd8b85344b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:24.555464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:24.556900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:24.557345 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:24.557702 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:24.560051 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:24.630173 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:24.656997 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:24.659805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:24.673936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1350/2699] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:29.515602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fec7b-ee55-45ca-82b0-e4b3284d5d5c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:29.516118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fec7b-ee55-45ca-82b0-e4b3284d5d5c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:29.516118 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fec7b-ee55-45ca-82b0-e4b3284d5d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:29.566569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-878fec7b-ee55-45ca-82b0-e4b3284d5d5c None None] MFA Rules not processed for user `5ba9373ffcab4af6bdda962a51c3e6c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:29.568900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-878fec7b-ee55-45ca-82b0-e4b3284d5d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:29.760240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1350/2700] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:29.774514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:29.779769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:29.879207 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd588ef-3eca-4700-8f04-d8c705f1aac5 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:29.885078 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f33d17889ee4fe3a9692abe24bbdea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:29.885979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:29.886434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:29.886703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:30.002386 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:30.003985 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:30.023664 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1351/2701] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:30.204931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:30.204931 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:30.205303 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:30.233731 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:30.235461 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:30.246083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:30.247620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:30.269550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645a53d-9a33-467c-8065-8d9978063242 service glance] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:30.276420 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:30.277356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:30.277645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:30.278363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] PATH_INFO: `/v3/users/ab248df5c41d4495b9c9f46ef33c1631` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:30.281561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab248df5c41d4495b9c9f46ef33c1631)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:30.282178 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:30.367244 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab248df5c41d4495b9c9f46ef33c1631'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.371593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1351/2702] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:21:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:30.377409 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab248df5c41d4495b9c9f46ef33c1631'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.387610 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab248df5c41d4495b9c9f46ef33c1631'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.388346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab248df5c41d4495b9c9f46ef33c1631'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.388947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab248df5c41d4495b9c9f46ef33c1631'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.439630 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:30.443962 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1352/2703] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:30 2022] DELETE /identity/v3/users/ab248df5c41d4495b9c9f46ef33c1631 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:21:30.444128 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:30.587632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081d912-5c5c-49bb-9f9c-4bb917590779 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:30.620072 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:30.691308 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:30.691951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:30.692224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:30.692671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:30.695908 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:30.734124 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:30.735595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:30.769199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:30.800577 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:30.802559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfd0b4-b54d-4557-9a12-a7231682a411 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:30.813286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1352/2704] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:21:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 17:21:30.859094 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ddfa6-407b-47dc-bc0e-a629aa6e637d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:30.866097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:30.867601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:30.868385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:30.868868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] PATH_INFO: `/v3/projects/250e4bed3c4c4c9c94a3143b2c6a7533` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:30.871510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250e4bed3c4c4c9c94a3143b2c6a7533)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:30.874680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:30.948311 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250e4bed3c4c4c9c94a3143b2c6a7533', 'request_id': 'req-b323430b-d63b-44e7-a4b7-f40b6e963f60'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.956402 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250e4bed3c4c4c9c94a3143b2c6a7533', 'request_id': 'req-b323430b-d63b-44e7-a4b7-f40b6e963f60'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.968123 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250e4bed3c4c4c9c94a3143b2c6a7533', 'request_id': 'req-b323430b-d63b-44e7-a4b7-f40b6e963f60'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.996638 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] The token cache is being invalidate because project 250e4bed3c4c4c9c94a3143b2c6a7533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:30.996936 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:30.999714 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-b323430b-d63b-44e7-a4b7-f40b6e963f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:31.004137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1353/2705] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:30 2022] DELETE /identity/v3/projects/250e4bed3c4c4c9c94a3143b2c6a7533 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:31.009834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.010546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.011091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.045822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:31.047540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:31.201135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1353/2706] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:31.210086 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.210579 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.211037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.238063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:31.239770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:31.257569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1354/2707] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:31.260714 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:31.284881 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38fd78-b213-4b2a-ae9d-4602a5899a94 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:31.302517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:31.303211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.303500 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.303791 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.304829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:21:31.305752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:31.307884 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:31.335861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1354/2708] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:21:31.340817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:31.360016 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55380-8984-4572-8e44-41cf79ac4b16 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:31.365450 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:31.366101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.366464 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.367029 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.367729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:31.369821 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:31.419703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1355/2709] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:31 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:31.424183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:31.447437 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c567ee-7f10-4df4-947d-a0b3cdec99aa admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:31.542390 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:31.594464 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:31.595214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.595562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.595900 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.596727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:31.599236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:31.639872 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1355/2710] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:31 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:31.645183 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:31.659809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323eee4-afda-43fe-ba4b-2942d98ef8cd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:31.676560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:31.678390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.679877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.681258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.683095 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:31.687246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:31.708728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1356/2711] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:31.714124 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:31.740632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4fa4b-eaf6-41e6-b763-fbf6de8e28a0 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:31.750120 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea794c4-46bf-4999-a408-28b11d3509df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:31.751087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea794c4-46bf-4999-a408-28b11d3509df admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.751426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea794c4-46bf-4999-a408-28b11d3509df admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.751797 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea794c4-46bf-4999-a408-28b11d3509df admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.752673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea794c4-46bf-4999-a408-28b11d3509df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:31.754761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea794c4-46bf-4999-a408-28b11d3509df admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:31.783280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1356/2712] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:31.788164 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:31.801508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2a793-07a6-466a-b0f5-7fa87d187454 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:31.809034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:31.809940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.810318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.810804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] PATH_INFO: `/v3/projects/e1141a0f81e94890a93c811444cfe9ec/users/1a35e631f1374b4b806ae251b81b3c0b/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:31.893367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1141a0f81e94890a93c811444cfe9ec, user_id=1a35e631f1374b4b806ae251b81b3c0b, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:31.900320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:31.959245 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1357/2713] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:31 2022] PUT /identity/v3/projects/e1141a0f81e94890a93c811444cfe9ec/users/1a35e631f1374b4b806ae251b81b3c0b/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:31.977808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:31.978895 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:31.979219 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:32.129328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] MFA Rules not processed for user `1a35e631f1374b4b806ae251b81b3c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:32.131341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:32.378580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1357/2714] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:32.407012 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:32.413070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:32.502601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:32.606195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c0c6e-1c8d-491b-a4e3-7b88a78ff64b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:32.614583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f3c1a-a8e1-4276-9b56-2f5d40e11acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:32.615424 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f3c1a-a8e1-4276-9b56-2f5d40e11acd service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:32.615980 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f3c1a-a8e1-4276-9b56-2f5d40e11acd service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:32.616482 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f3c1a-a8e1-4276-9b56-2f5d40e11acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:32.630550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f3c1a-a8e1-4276-9b56-2f5d40e11acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:32.633175 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f3c1a-a8e1-4276-9b56-2f5d40e11acd service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:32.645921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1358/2715] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:35.525561 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:35.527150 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:35.648527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628799c-6fcd-4cbd-b064-9c7d9c82929c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:35.660071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:35.660766 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:35.661051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:35.661328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] PATH_INFO: `/v3/users/68a610aeb01e4fc48871c8143c2e4c31` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:35.662541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a610aeb01e4fc48871c8143c2e4c31)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:35.665651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:35.762148 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a610aeb01e4fc48871c8143c2e4c31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:35.762634 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a610aeb01e4fc48871c8143c2e4c31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:35.772540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:35.789585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a610aeb01e4fc48871c8143c2e4c31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:35.805800 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a610aeb01e4fc48871c8143c2e4c31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:35.817778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a610aeb01e4fc48871c8143c2e4c31'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:35.825540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1358/2716] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:35 2022] DELETE /identity/v3/users/68a610aeb01e4fc48871c8143c2e4c31 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:35.830184 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:35.830871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:35.830871 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:35.857866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:35.859301 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:35.956878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1359/2717] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:35.970089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:35.971569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:36.072270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb6712-b339-4ea3-9b7c-9188df515e16 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:36.077536 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b726d16-3f23-497f-a540-741a67d95cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:36.078296 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b726d16-3f23-497f-a540-741a67d95cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:36.078576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b726d16-3f23-497f-a540-741a67d95cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:36.078850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b726d16-3f23-497f-a540-741a67d95cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:36.079908 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:36.081672 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:36.090700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b726d16-3f23-497f-a540-741a67d95cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:36.092374 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b726d16-3f23-497f-a540-741a67d95cd3 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:36.105147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1359/2718] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:36.110269 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978335f-6128-43f3-9c7a-0893641f92e3 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:36.120797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:36.121745 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:36.122143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:36.122536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] PATH_INFO: `/v3/users/1a35e631f1374b4b806ae251b81b3c0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:36.124477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a35e631f1374b4b806ae251b81b3c0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:36.126737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:36.218059 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a35e631f1374b4b806ae251b81b3c0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.258951 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a35e631f1374b4b806ae251b81b3c0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.281763 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a35e631f1374b4b806ae251b81b3c0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.283876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a35e631f1374b4b806ae251b81b3c0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.285477 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a35e631f1374b4b806ae251b81b3c0b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.346286 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1360/2719] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:36 2022] DELETE /identity/v3/users/1a35e631f1374b4b806ae251b81b3c0b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:21:36.354564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:36.354564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:36.354564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:36.395983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:36.403112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:36.524232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:36.525950 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:36.545598 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1360/2720] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:36.550978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:36.558541 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:36.560485 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:36.657401 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50409c1-3e81-4dcc-b593-2de5f4bef107 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:36.663750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:36.664616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:36.665242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:36.665645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] PATH_INFO: `/v3/projects/461f4d3a0a984e258de729b40739e47d` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:36.667483 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=461f4d3a0a984e258de729b40739e47d)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:36.670114 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:36.670777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890acd3b-497c-429b-a0f1-1c2c4c80cb9d None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:36.676483 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:36.682966 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:36.683322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:36.683604 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:36.708979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:36.714173 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:36.746718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1361/2721] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:36.796259 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461f4d3a0a984e258de729b40739e47d', 'request_id': 'req-cd713a98-820f-4e8c-8cbb-f2c31249a24f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.807253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461f4d3a0a984e258de729b40739e47d', 'request_id': 'req-cd713a98-820f-4e8c-8cbb-f2c31249a24f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.820948 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461f4d3a0a984e258de729b40739e47d', 'request_id': 'req-cd713a98-820f-4e8c-8cbb-f2c31249a24f'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.845586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] The token cache is being invalidate because project 461f4d3a0a984e258de729b40739e47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:36.846297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.847720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-cd713a98-820f-4e8c-8cbb-f2c31249a24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:36.868994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1361/2722] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:36 2022] DELETE /identity/v3/projects/461f4d3a0a984e258de729b40739e47d => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:37.332730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:37.337824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:37.363181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ab0dd-9d15-43d0-b30a-94789911b073 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:37.382259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:37.457113 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:37.458090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:37.458224 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:37.458481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] PATH_INFO: `/v3/projects/e1141a0f81e94890a93c811444cfe9ec` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:37.462211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1141a0f81e94890a93c811444cfe9ec)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:37.464089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:37.548028 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1141a0f81e94890a93c811444cfe9ec', 'request_id': 'req-8f3192aa-7d84-4272-b89f-5804d8d941a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:37.562681 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1141a0f81e94890a93c811444cfe9ec', 'request_id': 'req-8f3192aa-7d84-4272-b89f-5804d8d941a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:37.584622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1141a0f81e94890a93c811444cfe9ec', 'request_id': 'req-8f3192aa-7d84-4272-b89f-5804d8d941a6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:37.605387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] The token cache is being invalidate because project e1141a0f81e94890a93c811444cfe9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:37.605823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:37.606712 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8f3192aa-7d84-4272-b89f-5804d8d941a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:37.610919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1362/2723] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:37 2022] DELETE /identity/v3/projects/e1141a0f81e94890a93c811444cfe9ec => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:37.615726 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:37.616811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:37.617122 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:37.675810 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:37.677103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:37.815563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1362/2724] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:37.817274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:37.817564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:37.817982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:37.849043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:37.854981 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:37.872359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1363/2725] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:37.873896 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:37.887438 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7992-3c3a-4d56-9e18-c7f2dfcc9d44 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:37.892829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:37.893502 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:37.893988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:37.894417 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:37.895310 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:21:37.895920 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:37.898571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:37.917418 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1363/2726] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:21:37.919126 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:37.929581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01d3a-eef5-4c83-89cd-c1e5c94887ff None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:37.941400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:37.942379 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:37.942822 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:37.943398 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:37.945270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:37.947868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.005391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1364/2727] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:37 2022] POST /identity/v3/projects => generated 403 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:38.008335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.030921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fa33c-ad99-408e-b4ef-9e2bf8d7921d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.066455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:38.135338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.136006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.136318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.136597 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.137364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.139742 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.183113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1364/2728] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:38 2022] POST /identity/v3/users => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:38.184631 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.197409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1a933-1fde-4eae-94f9-94d3b8aaa716 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.202778 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.203407 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.203684 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.203957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.205067 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.206473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.218140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1365/2729] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:38.222687 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.239668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef2d58-9203-47df-8171-80c6f00398b1 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.248734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affaeb3b-5bfa-454d-9be9-5d47631e51ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.249230 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affaeb3b-5bfa-454d-9be9-5d47631e51ce admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.249539 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affaeb3b-5bfa-454d-9be9-5d47631e51ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.250239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affaeb3b-5bfa-454d-9be9-5d47631e51ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.250982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affaeb3b-5bfa-454d-9be9-5d47631e51ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.253348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affaeb3b-5bfa-454d-9be9-5d47631e51ce admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.273360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1365/2730] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:38.278274 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.293337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb99c20-8974-4de2-b7ed-c250263a7b23 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.304245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.305212 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.305555 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.306375 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] PATH_INFO: `/v3/projects/14884d11ffa24c6c8f577f5073fb00c6/users/6ac808350f76464aae024f6582c0cc5f/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.324761 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14884d11ffa24c6c8f577f5073fb00c6, user_id=6ac808350f76464aae024f6582c0cc5f, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.327569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.363946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1366/2731] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:38 2022] PUT /identity/v3/projects/14884d11ffa24c6c8f577f5073fb00c6/users/6ac808350f76464aae024f6582c0cc5f/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:38.377941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.378322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.378669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.429701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] MFA Rules not processed for user `6ac808350f76464aae024f6582c0cc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:38.431630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:38.542195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1366/2732] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:38.546985 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.610585 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:38.641581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b580d4d-0117-4592-a5c9-3f7ad51ebf3b admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.648004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.648768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.649139 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.649440 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.650104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.652047 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.689823 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1367/2733] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:38 2022] POST /identity/v3/projects => generated 403 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:38.693578 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.706020 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9f5a1-ac74-4ac5-afaa-462cb901f17e None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.792693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.793569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.793569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.793569 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.794446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.796501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.826340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1367/2734] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:38 2022] POST /identity/v3/users => generated 308 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:38.830818 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.843624 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba5cce-fa63-42f2-b20a-f36525b33092 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.849595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.850928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.851521 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.851948 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.852851 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.854677 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.868423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1368/2735] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:38.873350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.893324 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9443-181a-47a3-9ad3-d5a32e260c33 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:38.908697 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:38.909628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:38.909982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:38.910369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] PATH_INFO: `/v3/projects/873df03ec25a4a55af679dee6e655c37/users/e9ce1a6c0730403e8599e94bd6e2d3f9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:38.931050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873df03ec25a4a55af679dee6e655c37, user_id=e9ce1a6c0730403e8599e94bd6e2d3f9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:38.934555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:38.951161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1368/2736] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:38 2022] PUT /identity/v3/projects/873df03ec25a4a55af679dee6e655c37/users/e9ce1a6c0730403e8599e94bd6e2d3f9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:38.956318 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:38.967135 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a454485-46cf-46b4-b28d-b6ddcb1f1956 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:39.049529 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722af11-aa52-497a-9eb4-b471e1bd9e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:39.052306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722af11-aa52-497a-9eb4-b471e1bd9e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:39.052306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722af11-aa52-497a-9eb4-b471e1bd9e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:39.052306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722af11-aa52-497a-9eb4-b471e1bd9e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:39.052306 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722af11-aa52-497a-9eb4-b471e1bd9e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:39.053091 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722af11-aa52-497a-9eb4-b471e1bd9e14 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:39.067994 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1369/2737] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:39.069465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:39.077582 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:39.083818 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59166aaa-7e55-4b0d-8e0b-19e812c5d745 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:39.094351 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:39.095165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:39.095510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:39.095865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] PATH_INFO: `/v3/projects/873df03ec25a4a55af679dee6e655c37/users/e9ce1a6c0730403e8599e94bd6e2d3f9/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:39.098034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=873df03ec25a4a55af679dee6e655c37, user_id=e9ce1a6c0730403e8599e94bd6e2d3f9, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:39.102630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:39.115144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1369/2738] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:39 2022] PUT /identity/v3/projects/873df03ec25a4a55af679dee6e655c37/users/e9ce1a6c0730403e8599e94bd6e2d3f9/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:39.127883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:39.128181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:39.128347 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:39.169552 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] MFA Rules not processed for user `e9ce1a6c0730403e8599e94bd6e2d3f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:39.170676 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:39.274276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1370/2739] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:39.285618 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:39.291360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:39.389676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087d97e-64c5-46d8-8d9b-09932aeab3f4 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:39.394209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9929-99c3-460d-bdf1-ffca994efbc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:39.394852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9929-99c3-460d-bdf1-ffca994efbc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:39.395134 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9929-99c3-460d-bdf1-ffca994efbc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:39.395413 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9929-99c3-460d-bdf1-ffca994efbc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:39.409019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de9929-99c3-460d-bdf1-ffca994efbc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:39.410921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de9929-99c3-460d-bdf1-ffca994efbc9 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:39.425304 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1370/2740] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:39.615471 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:39.677982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:39.714535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf58f9f-81d3-49be-a6e3-b4ff3a676f9c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:39.720475 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:39.721176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:39.721463 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:39.721746 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] PATH_INFO: `/v3/users/6ac808350f76464aae024f6582c0cc5f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:39.723323 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac808350f76464aae024f6582c0cc5f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:39.724974 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:39.797858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac808350f76464aae024f6582c0cc5f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:39.798733 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac808350f76464aae024f6582c0cc5f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:39.820877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac808350f76464aae024f6582c0cc5f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:39.831289 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac808350f76464aae024f6582c0cc5f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:39.846626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac808350f76464aae024f6582c0cc5f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:39.858185 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1371/2741] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:39 2022] DELETE /identity/v3/users/6ac808350f76464aae024f6582c0cc5f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:39.862704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:39.863019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:39.863296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:39.886023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:39.887571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:39.968586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1371/2742] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:39.979348 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:39.981446 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:40.067042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e19d-cd84-4a1d-94e1-892a2beef39f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:40.072191 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:40.072982 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:40.073339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:40.073683 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:40.089545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:40.091581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:40.105951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1372/2743] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:40.452034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:40.453828 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:40.484140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12f6ef-d1fe-4c4f-985e-aa1d660f0fb1 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:40.496103 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:40.498197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:40.498759 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:40.499394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] PATH_INFO: `/v3/projects/14884d11ffa24c6c8f577f5073fb00c6` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:40.502861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14884d11ffa24c6c8f577f5073fb00c6)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:40.505549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:40.573459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14884d11ffa24c6c8f577f5073fb00c6', 'request_id': 'req-44a1e707-dd75-4fec-885a-9803c38ed8b9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.583241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14884d11ffa24c6c8f577f5073fb00c6', 'request_id': 'req-44a1e707-dd75-4fec-885a-9803c38ed8b9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.594280 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14884d11ffa24c6c8f577f5073fb00c6', 'request_id': 'req-44a1e707-dd75-4fec-885a-9803c38ed8b9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.626737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] The token cache is being invalidate because project 14884d11ffa24c6c8f577f5073fb00c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:40.626737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.626737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.626737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1372/2744] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:40 2022] DELETE /identity/v3/projects/14884d11ffa24c6c8f577f5073fb00c6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:40.629058 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:40.640287 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650202e1-29cc-4def-9678-6f6fd9367b8f service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:40.733015 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:40.733586 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:40.733874 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:40.734151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] PATH_INFO: `/v3/users/e9ce1a6c0730403e8599e94bd6e2d3f9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:40.735363 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ce1a6c0730403e8599e94bd6e2d3f9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:40.737341 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:40.743064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:40.838814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ce1a6c0730403e8599e94bd6e2d3f9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.839674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ce1a6c0730403e8599e94bd6e2d3f9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.857435 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ce1a6c0730403e8599e94bd6e2d3f9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.867944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ce1a6c0730403e8599e94bd6e2d3f9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.878551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-25c80217-e85e-49fa-94a1-4b83a4c02fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ce1a6c0730403e8599e94bd6e2d3f9'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:40.893373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1373/2745] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:21:40 2022] DELETE /identity/v3/users/e9ce1a6c0730403e8599e94bd6e2d3f9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:21:41.274442 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:41.276108 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:41.298822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1e707-dd75-4fec-885a-9803c38ed8b9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:41.326220 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:41.411708 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:41.413988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:41.414489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:41.414965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] PATH_INFO: `/v3/projects/873df03ec25a4a55af679dee6e655c37` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:41.417933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=873df03ec25a4a55af679dee6e655c37)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:41.421857 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:41.493598 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873df03ec25a4a55af679dee6e655c37', 'request_id': 'req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:41.507194 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873df03ec25a4a55af679dee6e655c37', 'request_id': 'req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:41.519364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873df03ec25a4a55af679dee6e655c37', 'request_id': 'req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:41.560238 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] The token cache is being invalidate because project 873df03ec25a4a55af679dee6e655c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:21:41.560551 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:41.561264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6cf1920f-e346-4c5e-ad27-7edd74b21fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:21:41.562891 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1373/2746] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:21:41 2022] DELETE /identity/v3/projects/873df03ec25a4a55af679dee6e655c37 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:41.570109 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c9637-7480-461d-a048-5d31bca57acc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:41.570809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c9637-7480-461d-a048-5d31bca57acc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:41.571478 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c9637-7480-461d-a048-5d31bca57acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:41.603453 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-169c9637-7480-461d-a048-5d31bca57acc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:41.604883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-169c9637-7480-461d-a048-5d31bca57acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:41.688309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1374/2747] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:41.697023 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:41.697996 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:41.698652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:41.734435 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:41.737446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:41.754813 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1374/2748] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:41.762728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c9637-7480-461d-a048-5d31bca57acc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:41.796302 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c9637-7480-461d-a048-5d31bca57acc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:41.810967 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:41.812036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:41.812538 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:41.813053 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:41.814381 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:21:41.814868 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:41.818051 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:41.856670 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1375/2749] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:21:41.862699 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:41.876769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd946fe-6a9f-454a-9a1a-deb3b893b62b None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:41.884425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:41.885414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:41.885717 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:41.885981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:41.886981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:41.889829 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:41.931750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1375/2750] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:41 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:41.936639 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:41.949429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e64f23-b2a3-45df-adea-924ec6233fb6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.031781 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.032439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.032701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.032998 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.033679 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.035707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.055814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:42.063387 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1376/2751] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:41 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:42.068838 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.082994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1327be-4dba-459c-96b9-d8ebfa6785d9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.091768 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.092734 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.093271 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.093754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.095053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.097730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.117758 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1376/2752] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:42.123101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.140131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375b493-0f9f-4689-b1ca-9436d3c8f324 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.148284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8313fa-f8c9-4eab-a7de-adbd3aac97a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.148912 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8313fa-f8c9-4eab-a7de-adbd3aac97a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.149226 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8313fa-f8c9-4eab-a7de-adbd3aac97a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.149503 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8313fa-f8c9-4eab-a7de-adbd3aac97a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.150214 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8313fa-f8c9-4eab-a7de-adbd3aac97a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.152054 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8313fa-f8c9-4eab-a7de-adbd3aac97a0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.164498 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1377/2753] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:21:42.169041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.181686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a170d1-56e6-495a-a5de-4c3e215f1f57 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.187550 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d2d72-fc90-418f-9a53-5e30af24ab3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.188246 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d2d72-fc90-418f-9a53-5e30af24ab3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.188529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d2d72-fc90-418f-9a53-5e30af24ab3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.188811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d2d72-fc90-418f-9a53-5e30af24ab3c admin admin] PATH_INFO: `/v3/projects/270b2a5475604c1a841f3a05ddd95f08/users/5941d1aac8514b948209384722c14af1/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.213636 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d2d72-fc90-418f-9a53-5e30af24ab3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=270b2a5475604c1a841f3a05ddd95f08, user_id=5941d1aac8514b948209384722c14af1, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.216952 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d2d72-fc90-418f-9a53-5e30af24ab3c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.234249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1377/2754] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:42 2022] PUT /identity/v3/projects/270b2a5475604c1a841f3a05ddd95f08/users/5941d1aac8514b948209384722c14af1/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:42.252121 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.252566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.252993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.297221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] MFA Rules not processed for user `5941d1aac8514b948209384722c14af1`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:42.299528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:42.311757 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.312342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.312342 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.333845 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] MFA Rules not processed for user `154a6b37687d471da649a8f6727d751e`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:42.335197 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:42.402346 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:42.422319 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1378/2755] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:42.427064 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1378/2756] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:21:42.428979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.518270 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aa938-268f-41bf-9df3-54c187023884 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.523727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.525863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.526269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.526649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.527609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.529856 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.571824 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1379/2757] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:21:42 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:21:42.572949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.584519 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4001b5-9321-4cbe-a3bf-dc5c99c91568 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.671567 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.672882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.673601 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.674110 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.676070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.682834 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.714969 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1379/2758] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:21:42 2022] POST /identity/v3/users => generated 294 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:21:42.719602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.733083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5852ea-5fe6-4bf6-9fb8-21bf743d4bca admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.737991 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.738606 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.738883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.739153 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.739880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.742391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.754152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1380/2759] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:21:42.759037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.771880 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2295831b-6094-4cfb-9be7-3749060339ce admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.777446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.778144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.778434 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.778704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] PATH_INFO: `/v3/projects/903bcaa49c4940d08000a97d19646ea6/users/1ddba17d8a1e40208afcab148dace525/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.797841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=903bcaa49c4940d08000a97d19646ea6, user_id=1ddba17d8a1e40208afcab148dace525, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.804253 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.823681 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1380/2760] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:42 2022] PUT /identity/v3/projects/903bcaa49c4940d08000a97d19646ea6/users/1ddba17d8a1e40208afcab148dace525/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:42.827662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.839177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b34b3-bb4a-4917-ab39-5988940d413a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.894423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:42.909421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45e724-c939-4ef0-b71d-c19ec642eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.910050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45e724-c939-4ef0-b71d-c19ec642eeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.910328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45e724-c939-4ef0-b71d-c19ec642eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.910602 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45e724-c939-4ef0-b71d-c19ec642eeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.911305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45e724-c939-4ef0-b71d-c19ec642eeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.913749 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45e724-c939-4ef0-b71d-c19ec642eeb0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.929581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1381/2761] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:21:42.931174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:42.941963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7ca70-7546-4f4f-bc7d-c6f039b57c8e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:42.947093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:42.947876 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.948291 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.948635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] PATH_INFO: `/v3/projects/903bcaa49c4940d08000a97d19646ea6/users/1ddba17d8a1e40208afcab148dace525/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:42.950739 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903bcaa49c4940d08000a97d19646ea6, user_id=1ddba17d8a1e40208afcab148dace525, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:42.953651 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:42.973263 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1381/2762] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:21:42 2022] PUT /identity/v3/projects/903bcaa49c4940d08000a97d19646ea6/users/1ddba17d8a1e40208afcab148dace525/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:21:42.979396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:42.979706 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:42.979965 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:43.011569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] MFA Rules not processed for user `1ddba17d8a1e40208afcab148dace525`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:43.013077 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:43.099290 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1382/2763] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:21:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:43.110912 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:43.112686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:43.177680 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dae6f9-c48a-4875-a6d6-7e53900c7d30 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:43.182379 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276885e-fa17-47eb-8f70-231de7560569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:43.183031 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276885e-fa17-47eb-8f70-231de7560569 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:43.183313 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276885e-fa17-47eb-8f70-231de7560569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:43.183586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276885e-fa17-47eb-8f70-231de7560569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:43.249907 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:43.253979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276885e-fa17-47eb-8f70-231de7560569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:43.257190 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276885e-fa17-47eb-8f70-231de7560569 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:43.270137 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1382/2764] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:47.956368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:47.971065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90059c4-6bf5-46a1-a115-1d6084e8f795 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:47.974773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:47.975528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:47.975528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:47.976190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:47.988201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:47.989819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:48.010527 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1383/2765] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:21:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:21:48.162223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:48.162223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:48.162223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:48.185909 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:21:48.187152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:48.265430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1383/2766] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:21:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:21:48.276690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:21:48.278339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:21:48.390545 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4b5a3-60fe-4926-b229-283734047d1b service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:21:48.405488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb8e4a-90d3-4b98-b95e-3cb59597c7d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:21:48.406198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8e4a-90d3-4b98-b95e-3cb59597c7d1 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:21:48.406353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8e4a-90d3-4b98-b95e-3cb59597c7d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:21:48.406592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb8e4a-90d3-4b98-b95e-3cb59597c7d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:21:48.421609 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb8e4a-90d3-4b98-b95e-3cb59597c7d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:21:48.423233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb8e4a-90d3-4b98-b95e-3cb59597c7d1 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:21:48.433626 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1384/2767] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:21:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:21:58.253523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=28 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:21:58.429537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=78 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:13.261684 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:13.326529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5ffd7-c9ea-40fe-b9a6-d0a7ac004cb5 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:13.331038 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:13.331685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:13.331967 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:13.332242 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] PATH_INFO: `/v3/users/5941d1aac8514b948209384722c14af1` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:13.334144 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5941d1aac8514b948209384722c14af1)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:13.336097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:13.384423 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5941d1aac8514b948209384722c14af1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.391174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5941d1aac8514b948209384722c14af1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.398991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5941d1aac8514b948209384722c14af1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.399621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5941d1aac8514b948209384722c14af1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.400296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5941d1aac8514b948209384722c14af1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.418055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1384/2768] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:13 2022] DELETE /identity/v3/users/5941d1aac8514b948209384722c14af1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:13.423019 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:13.423544 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:13.423643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:13.443280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:13.445061 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:13.512452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1385/2769] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:13.522811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:13.524620 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:13.582881 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed3143-148c-4150-a33c-5799f6802c7e admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:13.586711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:13.587322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:13.587596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:13.587864 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:13.596886 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:13.598389 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:13.608060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1385/2770] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:13.881133 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:13.882704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:13.897164 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89be69a-2bc3-4465-9e98-975208882b85 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:13.902718 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:13.903531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:13.903808 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:13.904078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] PATH_INFO: `/v3/projects/270b2a5475604c1a841f3a05ddd95f08` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:13.905295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=270b2a5475604c1a841f3a05ddd95f08)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:13.907201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:13.957570 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '270b2a5475604c1a841f3a05ddd95f08', 'request_id': 'req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.966331 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '270b2a5475604c1a841f3a05ddd95f08', 'request_id': 'req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.973165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '270b2a5475604c1a841f3a05ddd95f08', 'request_id': 'req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.986524 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] The token cache is being invalidate because project 270b2a5475604c1a841f3a05ddd95f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:13.986814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.987496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:13.989101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1386/2771] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:13 2022] DELETE /identity/v3/projects/270b2a5475604c1a841f3a05ddd95f08 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:13.993508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:14.004400 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7ac13-d594-4ce8-b77a-9ea68bcb100d service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:14.028790 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:14.091561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:14.092693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:14.093240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:14.093686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] PATH_INFO: `/v3/users/1ddba17d8a1e40208afcab148dace525` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:14.095764 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ddba17d8a1e40208afcab148dace525)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:14.100439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:14.171429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ddba17d8a1e40208afcab148dace525'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.180367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ddba17d8a1e40208afcab148dace525'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.192327 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ddba17d8a1e40208afcab148dace525'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.193866 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ddba17d8a1e40208afcab148dace525'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.194898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-1fe2515d-af8c-4cc2-aaa0-c32b2c606c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ddba17d8a1e40208afcab148dace525'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.218861 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1386/2772] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:13 2022] DELETE /identity/v3/users/1ddba17d8a1e40208afcab148dace525 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:14.512719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:14.515628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:14.530744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2187-a0cf-4528-86d3-cee6a7b9d87e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:14.614917 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:14.615619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:14.615904 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:14.616183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] PATH_INFO: `/v3/projects/903bcaa49c4940d08000a97d19646ea6` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:14.617596 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903bcaa49c4940d08000a97d19646ea6)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:14.619490 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:14.657811 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:14.679328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903bcaa49c4940d08000a97d19646ea6', 'request_id': 'req-9eb1afec-8626-4c58-8076-ef6edf067670'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.687114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903bcaa49c4940d08000a97d19646ea6', 'request_id': 'req-9eb1afec-8626-4c58-8076-ef6edf067670'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.694687 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903bcaa49c4940d08000a97d19646ea6', 'request_id': 'req-9eb1afec-8626-4c58-8076-ef6edf067670'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.705221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] The token cache is being invalidate because project 903bcaa49c4940d08000a97d19646ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:14.705511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.706192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-9eb1afec-8626-4c58-8076-ef6edf067670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:14.707713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1387/2773] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:14 2022] DELETE /identity/v3/projects/903bcaa49c4940d08000a97d19646ea6 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:14.713241 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:14.713737 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:14.713840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:14.734617 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:14.735867 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:14.796809 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1387/2774] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:14.805582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:14.805888 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:14.806165 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:14.825643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:14.826850 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:14.835001 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1388/2775] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:14.839368 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:14.849753 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1567b78-43a1-4c25-9038-9fe326ab0d32 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:14.854315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:14.854916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:14.855192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:14.855461 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:14.856174 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:22:14.856500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:14.858119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:14.870945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1388/2776] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:22:14.875473 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:14.884735 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b2939-ba7b-40b9-87d2-33041019222a None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:14.889430 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:14.890034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:14.890309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:14.890583 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:14.891247 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:14.893149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:14.922009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1389/2777] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:14 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:22:14.926505 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:14.941877 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72196aa5-6fb9-4170-8f78-47af31d5e66f admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:14.979690 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:15.017669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.018293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.018571 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.018846 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.019559 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.021914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.048370 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1389/2778] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:14 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:15.053742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.064114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6b263-25e3-4ee8-b3c3-5286fd24fce5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.069734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.070342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.070628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.070899 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.071593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.073213 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.084246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1390/2779] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:15.088557 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.099082 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31495a-195e-4dcd-b528-275e0b110c55 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.104177 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a5b92-62f7-4619-97e2-827d41106481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.104803 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a5b92-62f7-4619-97e2-827d41106481 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.105126 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a5b92-62f7-4619-97e2-827d41106481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.105403 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a5b92-62f7-4619-97e2-827d41106481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.106102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a5b92-62f7-4619-97e2-827d41106481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.107728 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a5b92-62f7-4619-97e2-827d41106481 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.119712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1390/2780] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:15.124501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.135480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b210-92fd-4bc9-8ddc-777693e7056a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.140152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.140829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.141140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.141419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] PATH_INFO: `/v3/projects/9bdf45ae048942438042f32beceeff02/users/00423225338e432aa2456822fb257e2f/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.158592 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf45ae048942438042f32beceeff02, user_id=00423225338e432aa2456822fb257e2f, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.163201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.186919 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1391/2781] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:15 2022] PUT /identity/v3/projects/9bdf45ae048942438042f32beceeff02/users/00423225338e432aa2456822fb257e2f/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:15.202008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.202292 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.202568 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.233874 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] MFA Rules not processed for user `00423225338e432aa2456822fb257e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:15.235056 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:15.309965 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1391/2782] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:15.314793 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.383329 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c42e4-2ff3-49bf-b50c-169a405ce6ac admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.388138 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.388767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.389083 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.389361 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.390043 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.391950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.429147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1392/2783] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:15 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:22:15.433219 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.445520 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccc3b1c-167d-425a-9c1a-78b14efa76f7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.506130 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.506811 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.507288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.507583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.508249 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.510199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.533642 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1392/2784] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:15 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:15.539132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.550278 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d540326-0b56-438e-8f77-81b2734c0f3f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.556066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.556675 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.556951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.557248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.557940 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.559523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.570566 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1393/2785] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:22:15.575489 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.586691 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5cf38-5f18-4dc3-8b47-ad4d1cddefd9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.591670 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.592380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.592661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.592937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] PATH_INFO: `/v3/projects/5c88f34575f6494ca6fd43d52594ddfe/users/517d2eec735e4724832349ab29dce812/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.609481 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c88f34575f6494ca6fd43d52594ddfe, user_id=517d2eec735e4724832349ab29dce812, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.612169 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.628902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1393/2786] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:15 2022] PUT /identity/v3/projects/5c88f34575f6494ca6fd43d52594ddfe/users/517d2eec735e4724832349ab29dce812/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:15.634366 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.647045 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de90d-f490-49f0-bf5a-a2b1b9e7871c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.671535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:15.715796 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22bea6-1565-4200-86a5-38feb6362bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.716575 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22bea6-1565-4200-86a5-38feb6362bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.716854 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22bea6-1565-4200-86a5-38feb6362bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.717159 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22bea6-1565-4200-86a5-38feb6362bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.717858 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22bea6-1565-4200-86a5-38feb6362bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.719480 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22bea6-1565-4200-86a5-38feb6362bf0 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.730506 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1394/2787] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:15.734751 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.741005 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:15.747465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868c793-8689-404a-9a4a-eaf7ef95c354 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.752978 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.753817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.754198 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.754479 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] PATH_INFO: `/v3/projects/5c88f34575f6494ca6fd43d52594ddfe/users/517d2eec735e4724832349ab29dce812/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.756761 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c88f34575f6494ca6fd43d52594ddfe, user_id=517d2eec735e4724832349ab29dce812, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.759446 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.770211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1394/2788] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:15 2022] PUT /identity/v3/projects/5c88f34575f6494ca6fd43d52594ddfe/users/517d2eec735e4724832349ab29dce812/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:15.780905 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.781299 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.781571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.811651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] MFA Rules not processed for user `517d2eec735e4724832349ab29dce812`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:15.812873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:15.884573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1395/2789] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:15.895773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:15.897634 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:15.957347 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328952e-11b6-4745-a202-6ef607cc1761 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:15.961258 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bb56b-a4b8-467f-8650-d4477c6a684d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:15.961882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bb56b-a4b8-467f-8650-d4477c6a684d service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:15.962163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bb56b-a4b8-467f-8650-d4477c6a684d service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:15.962441 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bb56b-a4b8-467f-8650-d4477c6a684d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:15.971792 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bb56b-a4b8-467f-8650-d4477c6a684d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:15.973264 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bb56b-a4b8-467f-8650-d4477c6a684d service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:15.983287 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1395/2790] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:16.054122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:16.117477 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e2e5c-4e2b-4642-bd6e-79e6254a8f4c None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:16.122447 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:16.123119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:16.123396 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:16.123664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] PATH_INFO: `/v3/users/00423225338e432aa2456822fb257e2f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:16.124843 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00423225338e432aa2456822fb257e2f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:16.126810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:16.173489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00423225338e432aa2456822fb257e2f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.174189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00423225338e432aa2456822fb257e2f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.175359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:16.188144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00423225338e432aa2456822fb257e2f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.197042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00423225338e432aa2456822fb257e2f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.203964 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00423225338e432aa2456822fb257e2f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.210364 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1396/2791] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:16 2022] DELETE /identity/v3/users/00423225338e432aa2456822fb257e2f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:16.214735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:16.215030 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:16.215302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:16.234371 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:16.235561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:16.298032 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1396/2792] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:16.308674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:16.310559 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:16.371201 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d5288-15f6-4a9c-bb50-2860afd7bf7f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:16.375110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:16.375717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:16.375993 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:16.376263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:16.385543 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:16.386999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:16.396614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1397/2793] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:16.699211 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:16.701097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:16.717661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915c053-cae8-4c08-8651-45cebc475d39 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:16.728921 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:16.729727 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:16.730006 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:16.730696 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] PATH_INFO: `/v3/projects/9bdf45ae048942438042f32beceeff02` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:16.732422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bdf45ae048942438042f32beceeff02)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:16.734154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:16.786048 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bdf45ae048942438042f32beceeff02', 'request_id': 'req-5e4ef246-3581-4aac-aaaf-dab2071c0294'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.793369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bdf45ae048942438042f32beceeff02', 'request_id': 'req-5e4ef246-3581-4aac-aaaf-dab2071c0294'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.798693 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bdf45ae048942438042f32beceeff02', 'request_id': 'req-5e4ef246-3581-4aac-aaaf-dab2071c0294'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.810775 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] The token cache is being invalidate because project 9bdf45ae048942438042f32beceeff02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:16.811063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.811683 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.813315 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1397/2794] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:16 2022] DELETE /identity/v3/projects/9bdf45ae048942438042f32beceeff02 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:16.817295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:16.826992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588dbe2-dbe8-4e76-8280-088012b622d1 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:16.882619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:16.883358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:16.883564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:16.883837 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] PATH_INFO: `/v3/users/517d2eec735e4724832349ab29dce812` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:16.885026 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] RBAC: Authorizing `identity:delete_user(user_id=517d2eec735e4724832349ab29dce812)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:16.886944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:16.932339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517d2eec735e4724832349ab29dce812'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.933004 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517d2eec735e4724832349ab29dce812'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.945880 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517d2eec735e4724832349ab29dce812'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.949423 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:16.953301 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517d2eec735e4724832349ab29dce812'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.962703 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-7eb787e6-6bc8-402e-a755-389f31a7b15a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517d2eec735e4724832349ab29dce812'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:16.968701 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1398/2795] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:16 2022] DELETE /identity/v3/users/517d2eec735e4724832349ab29dce812 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:17.245536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.246987 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:17.261982 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ef246-3581-4aac-aaaf-dab2071c0294 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.276777 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:17.324713 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.325305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.325590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.325869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] PATH_INFO: `/v3/projects/5c88f34575f6494ca6fd43d52594ddfe` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.327041 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c88f34575f6494ca6fd43d52594ddfe)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.328974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.378590 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c88f34575f6494ca6fd43d52594ddfe', 'request_id': 'req-460a041b-e8bc-479a-8c7c-b5e53915834a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:17.385706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c88f34575f6494ca6fd43d52594ddfe', 'request_id': 'req-460a041b-e8bc-479a-8c7c-b5e53915834a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:17.391540 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c88f34575f6494ca6fd43d52594ddfe', 'request_id': 'req-460a041b-e8bc-479a-8c7c-b5e53915834a'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:17.406436 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] The token cache is being invalidate because project 5c88f34575f6494ca6fd43d52594ddfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:17.406730 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:17.407487 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-460a041b-e8bc-479a-8c7c-b5e53915834a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:17.409019 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1398/2796] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:17 2022] DELETE /identity/v3/projects/5c88f34575f6494ca6fd43d52594ddfe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:17.414769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.415104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.415419 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.435686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:17.436920 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:17.498112 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1399/2797] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:17.505869 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.506155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.506427 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.526526 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:17.527725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:17.537336 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1399/2798] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:17.541456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.555275 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48593a1e-871d-42dd-b249-3e26cea69d98 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.559939 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.560551 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.560829 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.561129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.561828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:22:17.562129 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.563708 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.575916 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1400/2799] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:22:17.580522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.591232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fc8df-6985-42de-9da4-61eeed2dd660 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.595843 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.596445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.596722 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.597018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.597725 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.600555 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.632541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1400/2800] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:17 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:17.636380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.647192 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db090be-5be3-4511-a15d-5be7408e1d46 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.709131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.709944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.710228 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.710501 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.711181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.713141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.736720 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1401/2801] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:17 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:22:17.742522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.753113 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822983fc-ba0e-4839-b80f-6903eb2fe9ff admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.758462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.759363 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.759646 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.760089 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.760796 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.762586 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.774344 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1401/2802] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:17.779206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.791614 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c94fd-c874-4d0c-94ec-887efaf043a4 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.796309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51bf99-21fe-4126-99e3-afe4f83f51b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.796926 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51bf99-21fe-4126-99e3-afe4f83f51b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.797238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51bf99-21fe-4126-99e3-afe4f83f51b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.797512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51bf99-21fe-4126-99e3-afe4f83f51b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.798210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51bf99-21fe-4126-99e3-afe4f83f51b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.799882 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51bf99-21fe-4126-99e3-afe4f83f51b6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.812777 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1402/2803] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:17.817607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:17.827963 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df301ff-df45-42aa-85db-abe45f5031ce admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:17.832521 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:17.833235 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.833527 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.833797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] PATH_INFO: `/v3/projects/997df8b655104bd58f17eb81f21654a0/users/28d82329133c432bb65228777438699c/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.848128 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=997df8b655104bd58f17eb81f21654a0, user_id=28d82329133c432bb65228777438699c, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:17.851348 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:17.863914 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1402/2804] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:17 2022] PUT /identity/v3/projects/997df8b655104bd58f17eb81f21654a0/users/28d82329133c432bb65228777438699c/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:17.932342 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:17.932944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:17.932944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:17.962098 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] MFA Rules not processed for user `28d82329133c432bb65228777438699c`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:17.963638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:18.005514 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:18.030016 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1403/2805] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:18.034536 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.080584 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:18.101562 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac270f-85d4-4dc8-9364-98c74c4bc0fd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.106530 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.107154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.107437 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.107711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.108385 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.110335 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.141706 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1403/2806] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:18 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:18.143248 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.156671 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec9637-4023-40ca-bba7-566942a39542 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.212769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.213406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.213685 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.213957 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.214628 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.216518 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.237298 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1404/2807] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:18 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 17:22:18.241711 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.254645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73b53-3b95-4c3c-a156-d50845cacfcf admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.260553 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.261449 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.261805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.262040 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.262703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.264565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.277298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1404/2808] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:18.281576 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.292399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5c00-0947-4c50-973f-f0b9752bdbfe admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.296986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.297673 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.297950 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.298220 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] PATH_INFO: `/v3/projects/cb50b8e9805c444b92fb87f88542d7db/users/6861fea575744670a70d7e72a13419c4/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.319517 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb50b8e9805c444b92fb87f88542d7db, user_id=6861fea575744670a70d7e72a13419c4, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.322233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.336752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1405/2809] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:18 2022] PUT /identity/v3/projects/cb50b8e9805c444b92fb87f88542d7db/users/6861fea575744670a70d7e72a13419c4/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:18.340630 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.351222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d291a6-830a-47f4-9480-745c02052a35 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.409432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3efb89-b8d7-4650-9148-7ba35acd9355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.410060 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3efb89-b8d7-4650-9148-7ba35acd9355 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.410341 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3efb89-b8d7-4650-9148-7ba35acd9355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.410614 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3efb89-b8d7-4650-9148-7ba35acd9355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.411309 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3efb89-b8d7-4650-9148-7ba35acd9355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.412905 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3efb89-b8d7-4650-9148-7ba35acd9355 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.422797 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1405/2810] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:18.427814 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.438258 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a39d86e-c160-41fd-b5c9-5b90cf93d03c admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.443523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.444189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.444470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.444762 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] PATH_INFO: `/v3/projects/cb50b8e9805c444b92fb87f88542d7db/users/6861fea575744670a70d7e72a13419c4/roles/a05783d7bc51415a97b5a2545cdd7cc9` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.446910 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb50b8e9805c444b92fb87f88542d7db, user_id=6861fea575744670a70d7e72a13419c4, role_id=a05783d7bc51415a97b5a2545cdd7cc9)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.449662 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.462071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1406/2811] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:18 2022] PUT /identity/v3/projects/cb50b8e9805c444b92fb87f88542d7db/users/6861fea575744670a70d7e72a13419c4/roles/a05783d7bc51415a97b5a2545cdd7cc9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:18.472365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.472653 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.472931 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.506186 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] MFA Rules not processed for user `6861fea575744670a70d7e72a13419c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:18.507428 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:18.576712 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1406/2812] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:18.588230 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:18.589886 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:18.649535 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ee42a-a78a-4fad-a4cf-029772daaeba admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:18.653168 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af3e8d-815a-4cfd-a202-494540ba1154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:18.653782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3e8d-815a-4cfd-a202-494540ba1154 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:18.654086 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3e8d-815a-4cfd-a202-494540ba1154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:18.654359 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af3e8d-815a-4cfd-a202-494540ba1154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:18.664114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af3e8d-815a-4cfd-a202-494540ba1154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:18.665573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af3e8d-815a-4cfd-a202-494540ba1154 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:18.675686 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1407/2813] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:22:19.091491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:19.137993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:19.162447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b5bcb-aad6-44ed-a769-a498afc997d0 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:19.167639 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:19.168299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:19.168603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:19.168870 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] PATH_INFO: `/v3/users/28d82329133c432bb65228777438699c` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:19.170150 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d82329133c432bb65228777438699c)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:19.172303 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:19.227329 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d82329133c432bb65228777438699c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.235080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d82329133c432bb65228777438699c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.245002 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d82329133c432bb65228777438699c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.245709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d82329133c432bb65228777438699c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.246373 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d82329133c432bb65228777438699c'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.264949 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1407/2814] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:19 2022] DELETE /identity/v3/users/28d82329133c432bb65228777438699c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:19.269737 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11231-2c42-4056-9eae-773897d0552e None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:19.270322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11231-2c42-4056-9eae-773897d0552e None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:19.270322 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11231-2c42-4056-9eae-773897d0552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:19.290344 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-03f11231-2c42-4056-9eae-773897d0552e None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:19.291571 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-03f11231-2c42-4056-9eae-773897d0552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:19.328167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:19.360472 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1408/2815] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:19.371669 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:19.373384 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:19.434863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad676803-dd5a-44dc-b3de-8bccf69a80fd admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:19.438534 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:19.439154 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:19.439431 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:19.439701 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:19.450460 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:19.452152 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:19.463302 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1408/2816] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:19.736775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11231-2c42-4056-9eae-773897d0552e None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:19.738269 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-03f11231-2c42-4056-9eae-773897d0552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:19.753286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11231-2c42-4056-9eae-773897d0552e None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:19.758221 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:19.759622 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:19.759911 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:19.760198 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] PATH_INFO: `/v3/projects/997df8b655104bd58f17eb81f21654a0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:19.761540 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997df8b655104bd58f17eb81f21654a0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:19.763400 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:19.813316 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997df8b655104bd58f17eb81f21654a0', 'request_id': 'req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.821491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997df8b655104bd58f17eb81f21654a0', 'request_id': 'req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.827901 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997df8b655104bd58f17eb81f21654a0', 'request_id': 'req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.838786 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] The token cache is being invalidate because project 997df8b655104bd58f17eb81f21654a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:19.839071 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.839713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.841401 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1409/2817] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:19 2022] DELETE /identity/v3/projects/997df8b655104bd58f17eb81f21654a0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:19.845350 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:19.855865 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f93d1-319b-4d5a-b148-d46b95f35592 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:19.915039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:19.915720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:19.915995 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:19.916299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] PATH_INFO: `/v3/users/6861fea575744670a70d7e72a13419c4` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:19.917516 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6861fea575744670a70d7e72a13419c4)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:19.919432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:19.921192 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:19.984069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6861fea575744670a70d7e72a13419c4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.984069 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6861fea575744670a70d7e72a13419c4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.991517 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6861fea575744670a70d7e72a13419c4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.992299 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6861fea575744670a70d7e72a13419c4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:19.993039 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6861fea575744670a70d7e72a13419c4'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:20.016523 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1409/2818] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:19 2022] DELETE /identity/v3/users/6861fea575744670a70d7e72a13419c4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:20.249123 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.250700 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:20.312055 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.313933 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:20.314183 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e35a6b-25de-4a08-a175-4f1c5e2c02d6 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.378961 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.379499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.379819 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.380744 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.382553 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:20.393689 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03bc91-ff2f-4d23-afea-55604c7dd092 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.398974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.399637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.399918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.400193 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] PATH_INFO: `/v3/projects/cb50b8e9805c444b92fb87f88542d7db` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.401462 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb50b8e9805c444b92fb87f88542d7db)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.403405 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.418564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:20.454096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.454574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb50b8e9805c444b92fb87f88542d7db', 'request_id': 'req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:20.455532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca549c3-8bd9-4a81-8747-6f6f1ea0a24b service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.462478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb50b8e9805c444b92fb87f88542d7db', 'request_id': 'req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:20.468244 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1410/2819] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:20.469121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb50b8e9805c444b92fb87f88542d7db', 'request_id': 'req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:20.482707 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] The token cache is being invalidate because project cb50b8e9805c444b92fb87f88542d7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:20.482999 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:20.483807 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-bc0a3e6e-d6e1-4acc-8db7-afde1dc07dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:20.485623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1410/2820] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:20 2022] DELETE /identity/v3/projects/cb50b8e9805c444b92fb87f88542d7db => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:20.491409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.491692 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.491986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.513297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:20.514927 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:20.584030 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1411/2821] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:20.591882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.592168 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.592543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.613769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:20.615466 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:20.623760 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1411/2822] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:20.628172 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.638369 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb68e2-9411-42be-ad23-21a8a80df065 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.643111 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.643730 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.644009 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.644281 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.645006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:22:20.645835 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.647456 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.660404 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1412/2823] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:22:20.665603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.670377 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:20.677611 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0637dfa-ee78-48c9-b3d0-207ef0f11fcc None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.684589 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.685250 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.685529 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.685799 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.686474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.688802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.719875 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1412/2824] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:20 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:20.725406 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.735608 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99870e47-25eb-4ea8-9f41-37ee3d335ce3 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.804089 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.804707 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.805065 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.805380 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] PATH_INFO: `/v3/users` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.806176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.808486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.831798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1413/2825] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:20 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:20.836883 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.847554 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d5c7c-63ea-441f-aa5b-404009b2ad98 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.852848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.853510 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.853793 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.854070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.854772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.856399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.867053 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1413/2826] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:20.871147 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.883727 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d12b49-078c-4982-aa49-ad0822be030f admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.889140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c84314-6dfe-43c4-a0a6-5678e17a9677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.889813 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84314-6dfe-43c4-a0a6-5678e17a9677 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.890119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84314-6dfe-43c4-a0a6-5678e17a9677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.890460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84314-6dfe-43c4-a0a6-5678e17a9677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.891234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c84314-6dfe-43c4-a0a6-5678e17a9677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.893014 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c84314-6dfe-43c4-a0a6-5678e17a9677 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.905467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1414/2827] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:20.909622 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:20.920096 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f98ad-f51f-49b2-9bff-efd9009e506c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:20.924983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:20.925765 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.926080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.926358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] PATH_INFO: `/v3/projects/4b018a2a2c814f5e9932a294a1fb313f/users/6c74d188c62a47a8b00949b1185794a3/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:20.943099 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b018a2a2c814f5e9932a294a1fb313f, user_id=6c74d188c62a47a8b00949b1185794a3, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:20.946143 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:20.966480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1414/2828] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:20 2022] PUT /identity/v3/projects/4b018a2a2c814f5e9932a294a1fb313f/users/6c74d188c62a47a8b00949b1185794a3/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:22:20.979131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:20.979426 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:20.979809 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:21.018865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] MFA Rules not processed for user `6c74d188c62a47a8b00949b1185794a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:21.020108 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:21.042436 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:21.092894 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1415/2829] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:21.105045 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:21.106754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:21.172848 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af89c7-4c7e-400b-b741-9fe31d416aa7 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:21.176906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:21.177747 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:21.178034 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:21.178328 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:21.187841 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:21.189445 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:21.199988 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1415/2830] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:22:28.615006 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:28.616467 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:28.675690 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4fe80-0e40-4c37-a52a-755c542af5c6 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:28.680160 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:28.680788 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:28.681093 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:28.681368 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:28.683384 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:28.766748 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:28.768300 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7cca8-2790-438f-84f6-2e0aa08a352a service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:28.780496 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1416/2831] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:31.194947 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:31.574195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:31.636070 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c875-2eca-4f0f-a32f-198c8cbd51a2 service cinder] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:31.640613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:31.641556 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:31.642033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:31.642399 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] PATH_INFO: `/v3/users/6c74d188c62a47a8b00949b1185794a3` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:31.644276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c74d188c62a47a8b00949b1185794a3)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:31.646773 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:31.694806 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c74d188c62a47a8b00949b1185794a3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:31.702305 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c74d188c62a47a8b00949b1185794a3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:31.710994 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c74d188c62a47a8b00949b1185794a3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:31.711609 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c74d188c62a47a8b00949b1185794a3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:31.712209 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c74d188c62a47a8b00949b1185794a3'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:31.730805 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1416/2832] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:31 2022] DELETE /identity/v3/users/6c74d188c62a47a8b00949b1185794a3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:31.734947 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:31.735474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:31.735474 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:31.754655 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:31.755883 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:31.815145 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1417/2833] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:31.826119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:31.827625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:31.886591 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fac9db-bb93-4dd3-8ce6-843c15a3ddc0 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:31.890312 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6b5d9-7306-4acb-ae38-31d39e9b2f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:31.890941 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b5d9-7306-4acb-ae38-31d39e9b2f97 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:31.891222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b5d9-7306-4acb-ae38-31d39e9b2f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:31.891493 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b5d9-7306-4acb-ae38-31d39e9b2f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:31.902387 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6b5d9-7306-4acb-ae38-31d39e9b2f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:31.903839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6b5d9-7306-4acb-ae38-31d39e9b2f97 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:31.914902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1417/2834] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:32.205838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.207644 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:32.225523 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb1f45-d30a-40fc-8494-d391b00a63c8 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.230951 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.231664 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.231934 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.232280 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] PATH_INFO: `/v3/projects/4b018a2a2c814f5e9932a294a1fb313f` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.233528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b018a2a2c814f5e9932a294a1fb313f)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.235849 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.252362 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=100 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:32.302410 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b018a2a2c814f5e9932a294a1fb313f', 'request_id': 'req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:32.312171 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b018a2a2c814f5e9932a294a1fb313f', 'request_id': 'req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:32.321782 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b018a2a2c814f5e9932a294a1fb313f', 'request_id': 'req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:32.334409 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] The token cache is being invalidate because project 4b018a2a2c814f5e9932a294a1fb313f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:32.334768 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:32.335531 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-667c1f0e-9eeb-4f30-970a-d42a8fac2a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:32.337346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1418/2835] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:32 2022] DELETE /identity/v3/projects/4b018a2a2c814f5e9932a294a1fb313f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:32.342215 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.342740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.342840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.367577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:32.368915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:32.390508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:32.471511 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1418/2836] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:32.475225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.475225 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.475562 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.505842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:32.507136 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:32.517367 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1419/2837] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:32.522573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.536394 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31970e57-2e9c-4408-bd69-d261a54354b7 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.542974 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.545010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.545010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.557222 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.558093 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:22:32.558404 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.560133 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.585956 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1419/2838] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 17:22:32.591530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.604180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d14d5b-6c93-4fb3-abd4-b9e764cbe9b0 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.621177 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.622595 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.623078 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.623549 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.624790 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.628394 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.673580 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1420/2839] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:32 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:32.679859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.691119 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3830e-d2d7-4e1f-8e99-49cc8251309b admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.759656 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.760332 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.760623 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.760900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.761608 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.763585 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.788703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1420/2840] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:32 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:32.793169 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.807693 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6fa63-63fa-4afe-a749-ed6a47b48137 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.814066 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.814743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.815031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.815240 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.815946 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.817699 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.835651 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1421/2841] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:32.838333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.850603 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0d880-0ad0-4c9b-81b1-7dc2c66fb6b2 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.857097 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660cb140-c12c-4c0a-989d-3dc352984751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.857339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb140-c12c-4c0a-989d-3dc352984751 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.857508 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb140-c12c-4c0a-989d-3dc352984751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.857791 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660cb140-c12c-4c0a-989d-3dc352984751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.858501 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660cb140-c12c-4c0a-989d-3dc352984751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.860163 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660cb140-c12c-4c0a-989d-3dc352984751 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.878307 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1421/2842] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 17:22:32.882963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:32.897256 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bac79-d5b2-4358-97ec-675d0cc9fd0a admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:32.902452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:32.903152 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.903427 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.903704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] PATH_INFO: `/v3/projects/00a1957117fc4a8497f01474d163fae0/users/7f56196da2cd48dfb831994871e127be/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.924434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00a1957117fc4a8497f01474d163fae0, user_id=7f56196da2cd48dfb831994871e127be, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:32.928064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:32.944395 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1422/2843] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:32 2022] PUT /identity/v3/projects/00a1957117fc4a8497f01474d163fae0/users/7f56196da2cd48dfb831994871e127be/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:22:32.956852 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:32.957340 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:32.957616 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:32.997993 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] MFA Rules not processed for user `7f56196da2cd48dfb831994871e127be`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:32.999798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:33.069953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1422/2844] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:33.081751 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:33.084878 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:33.156232 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385b734-47ba-4759-8ce9-cd01007262c7 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:33.161491 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f163e4-1190-42ff-900e-410fc3c660af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:33.162125 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f163e4-1190-42ff-900e-410fc3c660af service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:33.162405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f163e4-1190-42ff-900e-410fc3c660af service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:33.162812 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f163e4-1190-42ff-900e-410fc3c660af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:33.173732 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f163e4-1190-42ff-900e-410fc3c660af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:33.175652 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f163e4-1190-42ff-900e-410fc3c660af service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:33.185902 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1423/2845] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:33.942918 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:33.944902 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:33.968139 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:34.017360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817d70e-3db3-4450-9784-2efa550b9c3d None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:34.021981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:34.022631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:34.022915 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:34.023300 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:34.025381 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:34.103293 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:34.104766 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:34.118945 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1423/2846] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:34.600452 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:34.601069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:34.601069 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:34.622728 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] MFA Rules not processed for user `7f56196da2cd48dfb831994871e127be`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:34.624031 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:34.632020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1424/2847] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:42.406752 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:42.408741 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:42.422414 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683d32a-7c33-4760-bd75-15e486999e9a service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:42.428478 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:42.429514 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:42.429927 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:42.430223 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] PATH_INFO: `/v3/users/d84cca5fc64f46109e9e62394da12437` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:42.431643 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84cca5fc64f46109e9e62394da12437)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:42.433546 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:42.501447 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84cca5fc64f46109e9e62394da12437'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.510065 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84cca5fc64f46109e9e62394da12437'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.518364 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84cca5fc64f46109e9e62394da12437'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.519025 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84cca5fc64f46109e9e62394da12437'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.520456 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84cca5fc64f46109e9e62394da12437'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.546625 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1424/2848] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:42 2022] DELETE /identity/v3/users/d84cca5fc64f46109e9e62394da12437 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:42.826656 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:42.839603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237d96-bda4-4c6c-b0f0-87b09740da05 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:42.886962 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:42.920325 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:42.921141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:42.921489 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:42.921773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] PATH_INFO: `/v3/projects/0a543d5eb6ef45569e57965779e4753e` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:42.923087 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a543d5eb6ef45569e57965779e4753e)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:42.925114 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:42.979742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a543d5eb6ef45569e57965779e4753e', 'request_id': 'req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.988944 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a543d5eb6ef45569e57965779e4753e', 'request_id': 'req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:42.996227 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a543d5eb6ef45569e57965779e4753e', 'request_id': 'req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.011512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] The token cache is being invalidate because project 0a543d5eb6ef45569e57965779e4753e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:43.011831 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.012533 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.014611 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1425/2849] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:42 2022] DELETE /identity/v3/projects/0a543d5eb6ef45569e57965779e4753e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:43.018682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:43.020804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:43.034587 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1e8e6-75e0-4072-9fdc-492c349ec0c9 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:43.210740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:43.234365 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:43.235036 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:43.235338 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:43.235645 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] PATH_INFO: `/v3/users/f2ce5a6cc58e456e8426968c4bfe3cfe` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:43.236882 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2ce5a6cc58e456e8426968c4bfe3cfe)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:43.239673 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:43.302740 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2ce5a6cc58e456e8426968c4bfe3cfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.312900 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2ce5a6cc58e456e8426968c4bfe3cfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.323200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2ce5a6cc58e456e8426968c4bfe3cfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.324142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2ce5a6cc58e456e8426968c4bfe3cfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.325109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-20732151-0c87-416b-9ab0-67bbfdd94428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2ce5a6cc58e456e8426968c4bfe3cfe'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.345333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1425/2850] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:43 2022] DELETE /identity/v3/users/f2ce5a6cc58e456e8426968c4bfe3cfe => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:43.544594 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:43.546042 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:43.561259 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027d3ec-c47c-44c6-a52b-e873b28e6bdf admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:43.628537 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:43.629963 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:43.630246 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:43.630649 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] PATH_INFO: `/v3/projects/61dbe2e4c2744463b6d76b38a8e6d881` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:43.631752 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=61dbe2e4c2744463b6d76b38a8e6d881)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:43.633729 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:43.694360 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61dbe2e4c2744463b6d76b38a8e6d881', 'request_id': 'req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.703689 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61dbe2e4c2744463b6d76b38a8e6d881', 'request_id': 'req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.712747 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61dbe2e4c2744463b6d76b38a8e6d881', 'request_id': 'req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.717665 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:43.727597 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] The token cache is being invalidate because project 61dbe2e4c2744463b6d76b38a8e6d881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:43.728234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.729210 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:43.731320 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1426/2851] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:43 2022] DELETE /identity/v3/projects/61dbe2e4c2744463b6d76b38a8e6d881 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:50.367142 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22843ec3-3f2e-472e-ad19-5d432c28fd5c None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:50.367682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22843ec3-3f2e-472e-ad19-5d432c28fd5c None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:50.367682 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22843ec3-3f2e-472e-ad19-5d432c28fd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:50.387358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-22843ec3-3f2e-472e-ad19-5d432c28fd5c None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:50.388595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-22843ec3-3f2e-472e-ad19-5d432c28fd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:50.453102 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1426/2852] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:22:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:50.496353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:50.498328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:50.576337 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253cb8d-0c3f-433b-b9cc-1c9e9fd3a48e admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:50.582488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:50.583233 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:50.583619 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:50.583921 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:50.593285 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:50.594979 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:50.604804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1427/2853] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:22:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:22:51.127593 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f238-01ba-4840-878e-1c0325cc5e7a None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:51.128050 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f238-01ba-4840-878e-1c0325cc5e7a None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:51.128439 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929f238-01ba-4840-878e-1c0325cc5e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:51.148075 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-1929f238-01ba-4840-878e-1c0325cc5e7a None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:51.149298 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1929f238-01ba-4840-878e-1c0325cc5e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:51.162195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1427/2854] 10.222.0.48 () {58 vars in 967 bytes} [Tue Jul 26 17:22:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:51.174579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:51.187442 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e13b76-3245-47e1-936e-4ffab43f809e service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:51.194130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:51.194754 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:51.195038 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:51.195314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:51.204184 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:51.205983 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:51.215076 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1428/2855] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:22:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:22:51.444004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cb9dc-bf78-4dd8-807b-058f85816444 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:51.444769 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cb9dc-bf78-4dd8-807b-058f85816444 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:51.444939 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489cb9dc-bf78-4dd8-807b-058f85816444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:51.465474 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-489cb9dc-bf78-4dd8-807b-058f85816444 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:51.466679 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-489cb9dc-bf78-4dd8-807b-058f85816444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:51.474649 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1428/2856] 10.222.0.48 () {58 vars in 1009 bytes} [Tue Jul 26 17:22:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:51.486745 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:51.497769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e53099-e43d-4c06-9c18-9f349ee44a9b service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:51.502848 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:51.503984 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:51.504279 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:51.504532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:51.513074 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:51.514719 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:51.523674 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1429/2857] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:22:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:22:51.864491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:51.864823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:51.865091 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:51.884033 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:51.885333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:51.892322 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1429/2858] 10.222.0.48 () {58 vars in 967 bytes} [Tue Jul 26 17:22:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 17:22:51.905034 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:51.916470 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fca485-10e8-45fe-bbba-f6dfdf77921d service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:51.921613 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf39815-f220-4a6e-bb6b-252120da77e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:51.922209 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39815-f220-4a6e-bb6b-252120da77e0 service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:51.922486 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39815-f220-4a6e-bb6b-252120da77e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:51.922757 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf39815-f220-4a6e-bb6b-252120da77e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:51.931493 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf39815-f220-4a6e-bb6b-252120da77e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:51.933099 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf39815-f220-4a6e-bb6b-252120da77e0 service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:51.941769 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1430/2859] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:22:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:22:54.081188 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:54.083661 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:54.133628 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:54.157471 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2afe9-a367-4efb-b3f3-610ef399130f None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:54.162432 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:54.163080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.163360 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.163635 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] PATH_INFO: `/v3/users/7f56196da2cd48dfb831994871e127be` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.164863 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f56196da2cd48dfb831994871e127be)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:54.166830 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:54.215079 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f56196da2cd48dfb831994871e127be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.221833 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f56196da2cd48dfb831994871e127be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.229495 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f56196da2cd48dfb831994871e127be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.230106 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f56196da2cd48dfb831994871e127be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.230709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f56196da2cd48dfb831994871e127be'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.248703 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1430/2860] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:22:54 2022] DELETE /identity/v3/users/7f56196da2cd48dfb831994871e127be => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:54.253209 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.254251 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.254373 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.274187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:54.275497 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:54.335032 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1431/2861] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:54.344748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:54.346200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:54.404121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97c26c-e94d-4ece-9274-e41683aaff5d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:54.408320 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07008350-1fea-4766-9646-e1ad0eb2dece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:54.408937 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008350-1fea-4766-9646-e1ad0eb2dece service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.409240 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008350-1fea-4766-9646-e1ad0eb2dece service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.409595 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008350-1fea-4766-9646-e1ad0eb2dece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.419430 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07008350-1fea-4766-9646-e1ad0eb2dece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:54.420853 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07008350-1fea-4766-9646-e1ad0eb2dece service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:54.430283 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1431/2862] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:54.605132 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:54.606573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:54.622305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cc8b3-d231-46b8-97da-43a2d5a391c9 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:54.627188 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:54.627861 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.628810 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.629141 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] PATH_INFO: `/v3/projects/00a1957117fc4a8497f01474d163fae0` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.630439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] RBAC: Authorizing `identity:delete_project(project_id=00a1957117fc4a8497f01474d163fae0)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:54.632499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:54.674328 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:54.683140 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00a1957117fc4a8497f01474d163fae0', 'request_id': 'req-67c81123-f3f8-45d2-85d5-a0872bf0a12f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.693588 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00a1957117fc4a8497f01474d163fae0', 'request_id': 'req-67c81123-f3f8-45d2-85d5-a0872bf0a12f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.700579 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00a1957117fc4a8497f01474d163fae0', 'request_id': 'req-67c81123-f3f8-45d2-85d5-a0872bf0a12f'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.713236 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] The token cache is being invalidate because project 00a1957117fc4a8497f01474d163fae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:22:54.713528 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.714167 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-67c81123-f3f8-45d2-85d5-a0872bf0a12f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:22:54.715678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1432/2863] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:22:54 2022] DELETE /identity/v3/projects/00a1957117fc4a8497f01474d163fae0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:54.720637 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.720929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.721288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.743480 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:54.744674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:54.807878 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1432/2864] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:54.815245 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.815539 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.815986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.836557 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:54.837798 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:54.845717 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1433/2865] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:54.850654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:54.860256 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d95e4-c4f4-412c-8c94-6c090ef63954 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:54.864984 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:54.865612 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.865893 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.866166 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] PATH_INFO: `/v3/domains` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.866859 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 17:22:54.867161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:54.868748 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:54.870288 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:54.882583 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1433/2866] 10.222.0.48 () {58 vars in 1153 bytes} [Tue Jul 26 17:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 17:22:54.886865 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:54.897264 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f478a-adbd-4477-9c8e-4122331a19bc None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:54.901923 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:54.902564 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:54.902841 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:54.903144 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:54.903847 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:54.905787 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:54.935238 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1434/2867] 10.222.0.48 () {60 vars in 1153 bytes} [Tue Jul 26 17:22:54 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:54.939453 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:54.949491 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0421351-20f9-4611-ae86-a401d0b75725 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:55.007922 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:55.008565 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:55.008840 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:55.009148 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] PATH_INFO: `/v3/users` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:55.009795 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:55.011698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:55.033594 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1434/2868] 10.222.0.48 () {60 vars in 1144 bytes} [Tue Jul 26 17:22:54 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 17:22:55.037828 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:55.047879 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d22e8-26a4-4010-8dc2-c66bfc4cc6e2 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:55.054546 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:55.055155 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:55.055466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:55.055743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:55.056434 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:55.058110 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:55.068765 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1435/2869] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:55.073043 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:55.083770 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc0400-36bc-4aaa-ba0f-16dff0fab908 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:55.088720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a3c27-420a-446f-9381-81a2e993e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:55.089422 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a3c27-420a-446f-9381-81a2e993e5ef admin admin] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:55.089709 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a3c27-420a-446f-9381-81a2e993e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:55.089981 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a3c27-420a-446f-9381-81a2e993e5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:55.090675 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a3c27-420a-446f-9381-81a2e993e5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:55.092281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a3c27-420a-446f-9381-81a2e993e5ef admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:55.102929 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1435/2870] 10.222.0.48 () {58 vars in 1122 bytes} [Tue Jul 26 17:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 17:22:55.107013 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:55.117704 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88362b63-9de5-45c2-b959-d2ffdb540740 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:55.122482 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:55.123127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:55.123402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:55.123672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] PATH_INFO: `/v3/projects/0b8c94d27e8f4ab29883ee3c27f0cd3c/users/873c9b425a534dc7aca0ef26a6147598/roles/94d6523ed4be482abd3484d9adce8e0b` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:55.138760 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8c94d27e8f4ab29883ee3c27f0cd3c, user_id=873c9b425a534dc7aca0ef26a6147598, role_id=94d6523ed4be482abd3484d9adce8e0b)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:55.141421 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:55.154672 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1436/2871] 10.222.0.48 () {60 vars in 1483 bytes} [Tue Jul 26 17:22:55 2022] PUT /identity/v3/projects/0b8c94d27e8f4ab29883ee3c27f0cd3c/users/873c9b425a534dc7aca0ef26a6147598/roles/94d6523ed4be482abd3484d9adce8e0b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:22:55.165063 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:55.165356 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:55.165633 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:55.194574 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] MFA Rules not processed for user `873c9b425a534dc7aca0ef26a6147598`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:22:55.195754 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:55.264561 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1436/2872] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:22:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:22:55.275999 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:55.277709 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:55.339261 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92382ace-5591-4abb-8eaa-126cea9e3482 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:55.343231 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:55.343842 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:55.344127 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:55.344565 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:55.357190 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:55.358643 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:55.369582 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1437/2873] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:58.807296 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:58.808677 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:58.865906 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f5b54-a02c-4e7f-b795-941329f2d892 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:58.869842 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:58.870459 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:58.870736 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:58.871004 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:58.873054 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:58.887155 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:22:58.949046 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:58.951685 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:58.966138 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1437/2874] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:22:59.683945 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:22:59.685402 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:59.697741 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fced25f-e984-4ca6-80a1-aea6593d8585 service cinder] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:22:59.701297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f2db6b70c241ec8bda9c96330f98c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:22:59.701896 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:22:59.702175 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:22:59.702465 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:22:59.704122 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:22:59.763560 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:22:59.765010 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df2657-4993-4c69-8768-e0cde975683a service cinder] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:22:59.774050 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1438/2875] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:22:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:23:02.547421 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:02.548898 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:02.560429 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcc553-b272-4690-b71c-0869421bae41 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:02.564020 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:02.564631 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:02.564910 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:02.565224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:02.567873 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:02.581663 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:02.583425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:02.594243 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1438/2876] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:23:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:23:09.769742 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:23:12.589470 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=32 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:23:12.945638 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b98205-8afa-47ba-8b78-c8b8462d2257 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:12.946193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b98205-8afa-47ba-8b78-c8b8462d2257 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:12.946193 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b98205-8afa-47ba-8b78-c8b8462d2257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:12.966755 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-03b98205-8afa-47ba-8b78-c8b8462d2257 None None] MFA Rules not processed for user `dcf968aeb53c4215927642e9931b4a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:23:12.968180 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-03b98205-8afa-47ba-8b78-c8b8462d2257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:12.976064 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1439/2877] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:23:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:23:17.943750 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:18.011944 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cf10e-18fe-43f8-abd1-7cd0f7e37a48 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:18.018817 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:18.019916 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:18.020187 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:18.020455 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] PATH_INFO: `/v3/users/873c9b425a534dc7aca0ef26a6147598` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:18.021659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=873c9b425a534dc7aca0ef26a6147598)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:18.023678 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:18.072160 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '873c9b425a534dc7aca0ef26a6147598'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.079822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '873c9b425a534dc7aca0ef26a6147598'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.088326 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '873c9b425a534dc7aca0ef26a6147598'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.089083 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '873c9b425a534dc7aca0ef26a6147598'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.089698 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '873c9b425a534dc7aca0ef26a6147598'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.107200 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1439/2878] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:23:17 2022] DELETE /identity/v3/users/873c9b425a534dc7aca0ef26a6147598 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:23:18.112343 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] REQUEST_METHOD: `POST` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:18.112838 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:18.112995 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:18.133383 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.auth.core [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] MFA Rules not processed for user `16f5c0d2019f40b59c659bd14cb74047`. Rule list: `[]` (Enabled: `True`). {{(pid=80462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:23:18.134605 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:18.194286 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1440/2879] 10.222.0.48 () {58 vars in 958 bytes} [Tue Jul 26 17:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:23:18.204195 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:18.205901 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:18.266425 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5f81b-e78d-468a-b133-779b67ca92c6 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:18.271109 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:18.271676 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:18.271953 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:18.272247 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:18.282724 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:18.284101 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:18.294080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1440/2880] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:23:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:23:18.590569 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:18.592405 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:18.611151 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3355d0-a25b-4d9f-88bf-45061e071b58 None None] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:18.620179 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:18.621176 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:18.621903 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:18.622335 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] PATH_INFO: `/v3/projects/0b8c94d27e8f4ab29883ee3c27f0cd3c` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:18.623734 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8c94d27e8f4ab29883ee3c27f0cd3c)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:18.627036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:18.686488 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8c94d27e8f4ab29883ee3c27f0cd3c', 'request_id': 'req-cdf40195-1c33-4f35-994b-bcd0288db937'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.697722 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8c94d27e8f4ab29883ee3c27f0cd3c', 'request_id': 'req-cdf40195-1c33-4f35-994b-bcd0288db937'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.705284 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8c94d27e8f4ab29883ee3c27f0cd3c', 'request_id': 'req-cdf40195-1c33-4f35-994b-bcd0288db937'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.718187 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] The token cache is being invalidate because project 0b8c94d27e8f4ab29883ee3c27f0cd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:23:18.718481 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.719149 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:23:18.720678 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1441/2881] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:23:18 2022] DELETE /identity/v3/projects/0b8c94d27e8f4ab29883ee3c27f0cd3c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:23:28.289733 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:23:28.714885 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:23:30.855613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:30.857210 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:30.868940 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e53a0-0814-44ea-a4dd-26722b8fe383 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:30.925077 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:30.925837 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:30.926227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:30.926607 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:30.928802 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:31.007021 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:31.008549 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:31.019815 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1441/2882] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:23:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:23:33.210578 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:33.212265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:33.227432 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf40195-1c33-4f35-994b-bcd0288db937 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:33.231429 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:33.232075 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:33.232358 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:33.232632 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:33.234593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:33.247346 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:33.248804 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:33.258018 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1442/2883] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:23:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:23:41.014552 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=50 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:23:43.253832 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=6 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:23:47.926804 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:47.928573 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:48.015543 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9547df3-5adc-499b-8245-4b40a67a7f97 service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:48.025819 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:48.026577 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:48.027080 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:48.029227 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:48.031541 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:48.050339 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:48.052199 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1552767e-c6a8-4173-a1db-fcaf41a52840 service nova] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:48.065318 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1442/2884] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:23:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:23:53.429386 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:53.430873 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:53.448887 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c2cb3-2727-4233-8f49-4d11b832b335 service neutron] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:53.454928 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a903f3771a62447d9c06fbeba414e9e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:53.455556 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:53.455834 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:53.456115 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:53.458088 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:53.471377 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:53.473222 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:53.484265 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1443/2885] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:23:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:23:54.425720 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] REQUEST_METHOD: `POST` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:54.426239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:54.426239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:54.477121 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.auth.core [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] MFA Rules not processed for user `564ffb348a9b4554ac2cedb7eced0bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 17:23:54.478358 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:54.542275 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1443/2886] 10.222.0.48 () {58 vars in 967 bytes} [Tue Jul 26 17:23:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 17:23:54.591816 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:23:54.593234 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:23:54.683511 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e17970-83f5-47cd-af7f-9a1b4b210fbd service nova] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:23:54.689276 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bfb7deb3aaf48a6992edb1b01b17aff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:23:54.689924 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:23:54.690206 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:23:54.690479 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:23:54.698860 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:23:54.700441 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:23:54.708953 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1444/2887] 10.222.0.48 () {60 vars in 1384 bytes} [Tue Jul 26 17:23:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 17:24:01.415744 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:24:01.417290 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:24:01.492111 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=100 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:24:01.497465 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c1163-e720-4961-94a4-9cb542fd3464 None None] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:24:01.502668 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:24:01.503333 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:24:01.503613 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:24:01.503890 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] PATH_INFO: `/v3/users/154a6b37687d471da649a8f6727d751e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:24:01.505135 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=154a6b37687d471da649a8f6727d751e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:24:01.507112 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:24:01.565426 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154a6b37687d471da649a8f6727d751e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.572894 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154a6b37687d471da649a8f6727d751e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.580868 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154a6b37687d471da649a8f6727d751e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.581652 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154a6b37687d471da649a8f6727d751e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.582251 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154a6b37687d471da649a8f6727d751e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.601297 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1444/2888] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:24:01 2022] DELETE /identity/v3/users/154a6b37687d471da649a8f6727d751e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:24:01.823399 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:24:01.838090 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0780a0d-0b1e-4701-9b49-c5be6192a40e service swift] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:24:01.902101 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:24:01.902743 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:24:01.903020 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:24:01.903295 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] PATH_INFO: `/v3/projects/6bc7b7b99e854517a3a43d0b7f7b471a` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:24:01.904439 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bc7b7b99e854517a3a43d0b7f7b471a)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:24:01.906391 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:24:01.957590 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bc7b7b99e854517a3a43d0b7f7b471a', 'request_id': 'req-ea738bd4-b463-438a-9d95-f2241c54bec5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.966131 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bc7b7b99e854517a3a43d0b7f7b471a', 'request_id': 'req-ea738bd4-b463-438a-9d95-f2241c54bec5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.968356 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:24:01.975600 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bc7b7b99e854517a3a43d0b7f7b471a', 'request_id': 'req-ea738bd4-b463-438a-9d95-f2241c54bec5'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.988189 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] The token cache is being invalidate because project 6bc7b7b99e854517a3a43d0b7f7b471a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:24:01.988469 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.989181 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:01.992713 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1445/2889] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:24:01 2022] DELETE /identity/v3/projects/6bc7b7b99e854517a3a43d0b7f7b471a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:24:01.995580 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:24:01.997273 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:24:02.011037 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75cc3b-c9b0-4439-af8b-09824e8dc21d admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:24:02.083159 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:24:02.083827 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:24:02.084107 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:24:02.084380 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] PATH_INFO: `/v3/users/5ba9373ffcab4af6bdda962a51c3e6c9` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:24:02.086704 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba9373ffcab4af6bdda962a51c3e6c9)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:24:02.089276 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:24:02.139007 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba9373ffcab4af6bdda962a51c3e6c9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.146458 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba9373ffcab4af6bdda962a51c3e6c9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.154367 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba9373ffcab4af6bdda962a51c3e6c9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.154979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba9373ffcab4af6bdda962a51c3e6c9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.155621 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba9373ffcab4af6bdda962a51c3e6c9'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.167018 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:24:02.175281 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1445/2890] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:24:01 2022] DELETE /identity/v3/users/5ba9373ffcab4af6bdda962a51c3e6c9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:24:02.358068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:24:02.360929 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:24:02.378096 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea738bd4-b463-438a-9d95-f2241c54bec5 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:24:02.437532 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:24:02.438932 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:24:02.439211 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:24:02.439581 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] PATH_INFO: `/v3/projects/2feec93864504689a1d5973c9f747bf4` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:24:02.440795 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2feec93864504689a1d5973c9f747bf4)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:24:02.442794 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:24:02.493130 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2feec93864504689a1d5973c9f747bf4', 'request_id': 'req-a7488bea-89f5-4b46-a5c8-2897e8895418'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.501767 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2feec93864504689a1d5973c9f747bf4', 'request_id': 'req-a7488bea-89f5-4b46-a5c8-2897e8895418'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.508573 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2feec93864504689a1d5973c9f747bf4', 'request_id': 'req-a7488bea-89f5-4b46-a5c8-2897e8895418'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.521554 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] The token cache is being invalidate because project 2feec93864504689a1d5973c9f747bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:24:02.521863 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.522499 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:24:02.524036 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1446/2891] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:24:02 2022] DELETE /identity/v3/projects/2feec93864504689a1d5973c9f747bf4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:24:12.176522 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=2 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:24:12.518775 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=55,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:25:18.410087 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:25:18.411798 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:18.473566 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ff910-661c-4345-95d0-78e17abb3a9c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:25:18.534500 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ed1fb62d8f3f4a73ac9d11a37c62f99f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:25:18.535262 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:25:18.535548 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:25:18.535822 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:25:18.537729 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:18.611735 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:25:18.613671 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:25:18.627316 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1446/2892] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:25:25.140119 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:25:25.142297 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:25.203224 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:25:25.206010 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:25.219011 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32818cf2-ab73-4427-9790-5c275a4125ca service neutron] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:25:25.223823 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:25:25.224469 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:25:25.224772 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:25:25.225232 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] PATH_INFO: `/v3/users/dcf968aeb53c4215927642e9931b4a5e` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:25:25.226390 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf968aeb53c4215927642e9931b4a5e)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:25:25.228659 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:25:25.235889 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488bea-89f5-4b46-a5c8-2897e8895418 admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:25:25.241844 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:25:25.242466 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] REQUEST_METHOD: `GET` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:25:25.242773 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:25:25.243044 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:25:25.245025 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:25.257862 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:25:25.259512 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:25:25.271897 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1447/2893] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:25:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 17:25:25.289161 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf968aeb53c4215927642e9931b4a5e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.296570 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf968aeb53c4215927642e9931b4a5e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.300991 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:25:25.306008 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf968aeb53c4215927642e9931b4a5e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.306654 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf968aeb53c4215927642e9931b4a5e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.307518 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf968aeb53c4215927642e9931b4a5e'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.328854 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1447/2894] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:25:25 2022] DELETE /identity/v3/users/dcf968aeb53c4215927642e9931b4a5e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:25:25.514530 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:25:25.527558 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c7089-b425-470d-8380-d1d3949781d8 service placement] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:25:25.587068 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:25:25.587715 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:25:25.587992 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:25:25.588268 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] PATH_INFO: `/v3/projects/b888b1053bb043dbb88d674c9a504a86` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:25:25.589460 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=b888b1053bb043dbb88d674c9a504a86)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:25:25.591353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:25:25.637960 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b888b1053bb043dbb88d674c9a504a86', 'request_id': 'req-427961c8-d7cc-43ff-9857-9baaeb01a0ed'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.647143 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b888b1053bb043dbb88d674c9a504a86', 'request_id': 'req-427961c8-d7cc-43ff-9857-9baaeb01a0ed'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.654267 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b888b1053bb043dbb88d674c9a504a86', 'request_id': 'req-427961c8-d7cc-43ff-9857-9baaeb01a0ed'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.666986 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] The token cache is being invalidate because project b888b1053bb043dbb88d674c9a504a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:25:25.667305 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.667968 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.669615 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1448/2895] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:25:25 2022] DELETE /identity/v3/projects/b888b1053bb043dbb88d674c9a504a86 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 17:25:25.674369 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:25:25.675785 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:25.689960 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4bb6-f092-4536-a0da-795269fae908 admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:25:25.751979 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:25:25.752686 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:25:25.752983 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:25:25.753279 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] PATH_INFO: `/v3/users/aee33ea192294139ae964b8a96daf83b` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:25:25.754454 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee33ea192294139ae964b8a96daf83b)` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:25:25.756396 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:25:25.805961 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee33ea192294139ae964b8a96daf83b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.813140 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee33ea192294139ae964b8a96daf83b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.820839 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee33ea192294139ae964b8a96daf83b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.821563 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee33ea192294139ae964b8a96daf83b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.822165 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.notifications [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee33ea192294139ae964b8a96daf83b'} {{(pid=80463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:25.841013 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1448/2896] 10.222.0.48 () {58 vars in 1224 bytes} [Tue Jul 26 17:25:25 2022] DELETE /identity/v3/users/aee33ea192294139ae964b8a96daf83b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:25:26.017382 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Authenticating user token {{(pid=80462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:25:26.018825 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.fernet_utils [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:25:26.034104 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427961c8-d7cc-43ff-9857-9baaeb01a0ed admin admin] Validating token access rules against request {{(pid=80462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:25:26.052339 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:25:26.096635 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16f5c0d2019f40b59c659bd14cb74047', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f28d3ad4783445e5a27ff264d5fb4f43', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:25:26.097309 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:25:26.097593 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:25:26.097876 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] PATH_INFO: `/v3/projects/f4b634e69a1b4a6d82b99a521aa639ef` {{(pid=80462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:25:26.099040 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b634e69a1b4a6d82b99a521aa639ef)` {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:25:26.100977 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] RBAC: Authorization granted {{(pid=80462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:25:26.152154 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b634e69a1b4a6d82b99a521aa639ef', 'request_id': 'req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:26.160353 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b634e69a1b4a6d82b99a521aa639ef', 'request_id': 'req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:26.166877 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b634e69a1b4a6d82b99a521aa639ef', 'request_id': 'req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b'} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:26.178314 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] The token cache is being invalidate because project f4b634e69a1b4a6d82b99a521aa639ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 17:25:26.178603 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:26.179263 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG keystone.notifications [None req-a7fad503-b56e-43d0-b7e5-7e5fcf2cb88b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 17:25:26.180774 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: [pid: 80462|app: 0|req: 1449/2897] 10.222.0.48 () {58 vars in 1233 bytes} [Tue Jul 26 17:25:26 2022] DELETE /identity/v3/projects/f4b634e69a1b4a6d82b99a521aa639ef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 17:25:35.837141 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:25:36.175327 dv-cinder-835384-17-iscsi devstack@keystone.service[80462]: DEBUG dbcounter [-] [80462] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 17:27:10.730391 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Authenticating user token {{(pid=80463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 17:27:10.732492 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:27:10.809644 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382333b5-41cd-4f23-8cce-71bb4f1e0c4c admin admin] Validating token access rules against request {{(pid=80463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 17:27:10.876596 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6dad20c291c5440ca6f7c89a3effe871', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8c34156723c24f918ca8fe84be2c7774', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 17:27:10.877277 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] REQUEST_METHOD: `GET` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 17:27:10.877560 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] SCRIPT_NAME: `/identity` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 17:27:10.877831 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 17:27:10.879674 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.fernet_utils [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 17:27:10.892457 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 17:27:10.894239 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b209dd0-8bb9-4fca-8403-3ca12a91dfb1 service placement] RBAC: Authorization granted {{(pid=80463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 17:27:10.907171 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: [pid: 80463|app: 0|req: 1449/2898] 10.222.0.48 () {60 vars in 1365 bytes} [Tue Jul 26 17:27:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 17:27:20.900538 dv-cinder-835384-17-iscsi devstack@keystone.service[80463]: DEBUG dbcounter [-] [80463] Writing DB stats keystone:SELECT=58 {{(pid=80463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}